DL Classe 0 - You can do it
•
1 like•350 views
Here are some key terms that are similar to "champagne": - Sparkling wines - French champagne - Cognac - Rosé - White wine - Sparkling wine - Wine - Burgundy - Bordeaux - Cava - Prosecco Some specific champagne brands that are similar terms include Moët, Veuve Clicquot, Dom Pérignon, Taittinger, and Bollinger. Grape varieties used in champagne production like Chardonnay and Pinot Noir could also be considered similar terms.
DL Classe 0 - You can do it
- 1. Louis Monier @louis_monier https://www.linkedin.com/in/louismonier A Taste of Deep Learning You can do it ! Gregory Renard @redo https://www.linkedin.com/in/gregoryrenard Class 0 - Q1 - 2016
- 2. The Big Bet Credit: Google Trends
- 3. Why we are Bullish about the Future ...100 years... ...10 years... ??? ...100 years?...
- 4. Projections McKinsey projects $50T by 2025 - Knowledge work: big data, smart assistants, marketing, sports… - Finance, health care, genomics, law, insurance... - IoT: smart appliances, sensors, cameras, wearables - Robots, autonomous vehicles, drones, manufacturing … so VCs are pouring money into Deep Learning companies.
- 5. A Brief History of Artificial Intelligence
- 6. How to define Artificial Intelligence? Easy for People (intelligence) Vision, speech, understanding language Planning, common sense Having a conversation Reading emotions Translating Russian to English Playing Chess, Go Proving math theorems Easy for Machines (stamina) Multiplying huge matrices Searching large databases Sorting a trillion records Finding a path in a huge maze Hard for People and Machines Predicting the weather Predicting the winning lottery ticket
- 7. 1770 The Mechanical Turk The “Turing Test”1950 1956Dartmouth Conference Checkers-playing program “Magic” 1957 Perceptrons, Frank Rosenblatt 1966ELIZA, first chatterbot (*) 1969Minsky and Papert Lighthill Report1973 First AI Winter $$$ 1958 LISP
- 8. 1980 1981 1984 Second AI Winter Cyc Expert systems Expert systems disappoint, end of 5th gen 5th Generation Computing, Japan $$$ 1982 Connection Machine (#$isa #$BarakObama #$USPresident) (#$genls #$Tree-ThePlant #$Plant) ... 1989 1997Deep Blue vs Kasparov 1998 Google 1995 AltaVista Hand-written digits recognition BabelFish 1991 http://www
- 9. 2006 2009 2010 2011 2012 2013 2015 Personal Assistant (Siri, Angie) Fundamental DL technique cracked! The Unreasonable Effectiveness of DataVoice breakthrough IBM Watson wins at Jeopardy Image breakthrough Deep Mind plays Atari games Text breakthrough TensorFlow Google Photos 2004Project DAVE autonomous robot Google Brain
- 10. So what did we learn?
- 11. Lesson: People are easy to fool! Chess: large-scale search Wikipedia, tricks, fast search No real ‘understanding”, lots of data, lots of computing power Eliza: 100% smoke and mirrors. Addictive.
- 12. Lesson: Some Things are Particularly Hard to Code Vision Natural Language Speech He said that she replied that they could not agree. But she was wrong.
- 13. Use Rules? Use Data? Option 1: Write a set of rules - Use logic and heuristics to assemble hand-coded rules - The real world is messy and changes all the time - This approach won't scale, and it’s expensive (people) - (We don’t have a symbol-manipulation engine in our heads) Option 2: Learn from data - Uses examples to find patterns automatically: the more data, the better! - Automatically adapts to new data - This approach scales, and it’s cheap (computing power)
- 14. Conclusion? More data and computing power always beat fancy algorithms!
- 15. Lesson: Some Problems are AI-Complete “...I got a .45 and a shovel...”
- 16. Lesson: Some Problems are AI-Complete “...I got a .45 and a shovel...” “Let’s see: a .45 is a gun, and a shovel is used to dig holes. A father is usually very protective of his daughter, and he looks intensely at the daughter's date when he says that. Most likely interpretation: if anything happens to his daughter (accident, pregnancy), he will kill the guy with the gun and bury him with the shovel. It’s a threat, but it can’t be serious, this is illegal, and he is talking in front of his daughter; so it’s a funny threat, a warning. The guy will still understand that the dad means business and expects him to take good care of his daughter.” ... Let’s have a quick chuckle, and on to the next line…
- 17. Examples of AI-Complete Tasks Vision Natural Language Understanding - Say: “Wreck a nice beach” - Now: “Recognize speech” Automated Translation Virtual Assistant (Personal Assistant) Holding a conversation Truly summarizing text Dealing with the real world - Navigation - Planning - Adapting to new environments
- 18. Lesson: Don’t be afraid to Experiment As we’ll learn: turn everything into numbers, mix them happily! Don’t try to follow rigorously what is going on! Don’t expect a mathematical proof for everything!
- 19. If you remember only one thing... The approach that won: learn from the data!
- 20. Interlude: A.I. Movies and Memes
- 21. 2001 - A Space Odyssey
- 22. Terminator
- 24. Sarah - Eureka
- 25. Her
- 26. Transcendence
- 27. Ex Machina
- 28. 1/ Artificial Narrow Intelligence (ANI) = Weak IA 2/ Artificial General Intelligence (AGI) = Strong IA 3/ Artificial SuperIntelligence (ASI) = 12k IQ Art & Conception of AI Today, computers are not smart, all the ingenuity of the designers in AI is to make you think they are reproducing human thought patterns. The Future of AI: What if We Succeed?
- 29. Take a Break!
- 30. A Taste of Deep Learning
- 31. What is Machine Learning? = ? question response prediction 1. Training a model question prediction 2. Using the trained model (Supervised learning)
- 32. Let’s do a (totally fake) Clinical Study Patient id #packs per day #hours exercise per day heart problem 1 .5 2 0 2 2 0 1 3 3 6 1 4 0 5 0 5 1 0 1 6 1.5 5 0 Can we learn a pattern? Can we use it to predict outcome of new patients?
- 33. Linear Regression Smoking Exercising Heart problem Healthy heartWhat about Sally? Compute a linear combination of - p = # packs per week - h = # hours of exercise per week Find w1, w2, w3 that best match the data. That’s the learning. if (score > 0) then predict heart problem. Simple statistics. Score > 0 Score < 0
- 34. How to find the best values for w1, w2 and w3 ? Parameter Error a b Define error = |expected - computed| 2 Find parameters that minimize average error. Perform Gradient Descent: quality goes up. take a step downhill
- 35. Why “neurons”? + x w1 p x w2 h w3 Biological Neuron Artificial Neuron ChoiceScore Linear Combination Nonlinearity
- 36. Demo: ConvNetJS Credits and thanks to Andrej Karpathy, Stanford https://github.com/holbertonschool/deep-learning/tree/master/ConvNetJS
- 37. You just trained your first model!
- 39. Limitations of a Linear Classifier Food BMI Not so much Healthy starve binge skinny pudgy y x Bad Good
- 41. Demo: let’s make an ensemble!
- 42. More Layers: Deep Learning input1 input2 input3 input layer hidden layers output layer circle iris eye face 89% likely to be a face
- 43. Demo: let’s go deep!
- 44. Holy Moly! You just trained your first deep model !!!
- 45. Why does this work? Neural networks with at least one hidden layer can approximate any reasonably smooth function. Large networks have lots of solutions (minima), most of them very good. Gradient descent is very simple and very powerful.
- 46. Everyday examples of Deep Learning Structured data: Netflix, Spotify and YouTube recommendations; Amazon suggestions; CC fraud detection Text: Spam filtering; good spelling suggestions; matching ad to content; automated translation Images: Google Photos; search by image; FaceBook face tagging; thumbnail for YouTube videos; OCR and handwriting recognition; surveillance videos Voice: Android voice input; Nuance (Siri); transcription Combo: Autonomous vehicles (soon); Virtual Assistant, Industrial robots
- 48. Other Forms of Machine Learning
- 49. Unsupervised Learning Automatically learn structure in data Clustering More compact representation Semi-supervised learning Credit: http://colah.github.io/posts/2014-10-Visualizing-MNIST/
- 50. Reinforcement Learning Learn not from static data, but from interacting with a system - playing a game - flying a plane - driving a car - learning a task System do something get a score
- 52. Learning to flip pancakes Credit: Italian Institute of Technology
- 53. Stanford Helicopter - Andrew Ng
- 54. Driverless Car
- 55. Deep Mind plays Atari Games
- 56. DeepMind AlphaGo Go is much harder than chess Oct 2015: AlphaGo beats Fan Hui, a top Go player Jan 2016: paper comes out, world goes wild Match with Lee Sedol, #1 player, in March 2016
- 57. If you remember only one thing... We build a model from a set of examples. It starts as a random set of parameters. We measure how well the predictions match the truth. We tweak the parameters to improve this match. A good model will generalize to new data, making useful predictions.
- 58. What we will learn How Deep Learning works, more precisely. What all the terms mean. It’s a big zoo. Using existing frameworks: TensorFlow, Torch... Downloading, using, modifying existing models. All the tricks to tune models. Mapping a problem to an architecture.
- 59. Take a Break!
- 60. A Taste of Deep Learning for Images
- 61. A Brief History of Visual Recognition 2012 - Annus Mirabilis for DL. ImageNet contest. Alex Krizhevsky, Ilya Sutskever, Geoffrey Hinton, University of Toronto
- 62. Why is Vision Hard?
- 63. Rules really don’t work for vision I’m asking you to describe cherry blossoms. Please use precise features and rules! “Well, white petals arranged in a circle. Unless some of the petals have fallen. With little white sticks and black dots arranged like this. Oh, except if seen from the side. Or if they overlap. Or if the sun is behind. Ignore the bee…” Next task: a human face. Any human face. Very slow progress, even with generations of graduate students.
- 64. Demo: Google Photos Face Recognition
- 68. Glasses Hat Face paint Tiny Overexposed Sleeping Eating
- 69. Demo: Google Photos Entity Recognition
- 70. horse horse horse horse lake cup cup cup
- 71. boat snail helicopter mushroom frog chairbed bison insect legos apple churchfountain apple mouse stuffed toy
- 72. Demo: Google Photos Scene Recognition
- 73. bowlingscuba green sunset dancing halloween birthdaysleeping tennis lunch adventure drinking happiness birthday formal wear
- 74. How did they do that? Deep Neural Networks Years of Google Images, tagged by users, as training set Face recognition is a special thing
- 75. Demo: FaceYou
- 76. Demo: ParotAI
- 77. NeuralTalk Video
- 78. Running it in Reverse
- 82. Deep Dream by Google
- 84. J.M.Turner, 1805 Kandinsky, 1913 Pablo Picasso, 1910 Edvard Munch, 1893Vincent van Gogh, 1889 Credit: http://arxiv.org/pdf/1508.06576v1.pdf
- 85. Demo: Deep Art live
- 87. Image Captioning Source : http://googleresearch.blogspot.com/2014/11/a-picture-is-worth-thousand-coherent.html
- 88. What we will learn Convolutional Neural Networks (ConvNets) Adapting existing models
- 89. If you remember only one thing... Vision is not a task that can be reduced to simple rules. Immense progress since modern ConvNets and GPUs, ~2012. Many real-life applications today. Expect a lot more.
- 90. Take a Break!
- 91. A Taste of Deep Learning for Text
- 92. Natural Language Processing (NLP) Parsing words Spell checking Finding synonyms Part of Speech (POS) Tagging Classification - Encoding and language detection - Sentiment analysis - Spam detection - Matching ad to content Extracting entities (people, places…) Full-text Search Summarization Automated Translation Question Answering Virtual Assistant - Siri, Watson, Alexa, … QA - Her, HAL, Sarah, … Empathy & Emotion
- 93. What is so hard about Natural Language Processing?
- 94. Understanding H2H Communication Source : https://mishahlini1996.wordpress.com
- 95. Languages are Complex - Ambiguity Jaguar or Jaguar ?
- 96. Languages are Complex - Context “The Jaguar eats his prey” => predator => big cat “The Jaguar eats the road” => image => car Also: idioms, technical lingo, slang, humor, sarcasm, poetry, emotions...
- 97. Unsaid, but implied “The baby looks happy!” ... “Where are the cookies?”
- 98. Word and Sentence Similarity
- 99. Semantic Distance for Words cat purring sofa New York dogkitten lion Not to scale :) serendipity less related 2348883608 furball feline hat fur turkish angora nanocrystals
- 100. Terms similar to Champagne french champagne, cognac, champagne's, champagnes, veuve clicquot, cremant, louis roederer, rosé, taittinger, fine champagne, champagne wine, sparkling wines, dom pérignon, dom perignon, pol roger, vintage champagne, bubblies, pommery, rose wine, pink wine, blancs, french wine, cliquot, beaujolais nouveau, sancerre, sparkling, burgundy, chateau, chablis, cognacs, pink champagne, domaine, moët, methode champenoise, burgundy wines, apéritif, armagnac, chandon, champenoise, beaujolais, heidsieck, marnier, wine, bourgogne, aperitif, chateau margaux, demi-sec, moelleux, champagne cocktail, crémant, half-bottle, cuvée, brut, ruinart, champagne flute, st emilion, white wine, loire valley, wine cocktail, veuve, drinking champagne, french wines, blanc, chardonnay wine, champagne glass, cuvées, mauzac, roederer estate, laurent- perrier, puligny, negociant, prosecco, rose wines, gloria ferrer, red wine, musigny, coteaux, corton-charlemagne, fine wine, dessert wine, bordeaux, champagne glasses, cheval blanc, champagne flutes, cuvees, champange, four wines, montlouis, rémy martin, primeur, fine wines, lirac, d'yquem, burgundy wine, red bordeaux, brandy, cuvee, white burgundy, chardonnay, chambolle-musigny, cheverny, great vintages, yquem, special wines, wonderful wines, burgundies, half bottles, grand marnier, grand cru, primeurs, sauterne, minervois, pouilly-fuissé, sauternes, chambertin, white bordeaux, vougeot, epernay, vin gris, chalonnaise, quaffer, loire, sweet white wine, d'aunis, côtes, gevrey-chambertin, limoux, english wine, chateaux, château haut-brion, blanche, pinot meunier, six glasses, mâconnais, épernay, bourbon, sparkler, volnay, white wines, chassagne-montrachet, burgundys, vin jaune, claret, beaune, grande champagne, white grapes, bordeaux wine, dessert wines, crème de cassis, pinot noir grapes, chardonnay grapes, armand de brignac, select wines, calvados, country wine, muscadet, leflaive, reisling, cointreau, own wine, caveau, clos de vougeot, inexpensive wines, vosne-romanée..., expensive wines, red burgundy, barsac, delicious wine, wine flight, puligny-montrachet, rousanne, châteauneuf-du-pape, liqueur, schramsberg, touraine, montrachet, arbois, lanson, vintage wine, chateauneuf, blanquette, non-vintage, orange wine, three wines, wine.the, banyuls, merlot wine, vendange, red table wine, sweet wines, santenay, languedoc, moscato d'asti …
- 101. Terms similar to Brad Pitt angelina jolie, george clooney, cameron diaz, julia roberts, leonardo dicaprio, matt damon, tom cruise, nicole kidman, reese witherspoon, charlize theron, jennifer aniston, halle berry, kate winslet, jessica biel, ben affleck, bruce willis, scarlett johansson, uma thurman, matthew mcconaughey, jake gyllenhaal, sandra bullock, oscar winner, gwyneth paltrow, sean penn, demi moore, naomi watts, colin farrell, mickey rourke, orlando bloom, bradley cooper, natalie portman, jennifer garner, tom hanks, dicaprio, jessica chastain, robert de niro, julianne moore, leo dicaprio, channing tatum, kirsten dunst, jessica alba, emily blunt, salma hayek, ryan gosling, mark wahlberg, renee zellweger, drew barrymore, renée zellweger, gerard butler, hilary swank, ryan phillippe, john malkovich, nicolas cage, kate hudson, sharon stone, sienna miller, new movie, kim basinger, robert downey jr, keira knightley, ryan reynolds, johnny depp, jennifer connelly, edward norton, emma stone, don cheadle, marisa tomei, jason statham, eva mendes, kate beckinsale, oscar-winner, katie holmes, kelly preston, denzel washington, zac efron, clive owen, oscar-winning, forest whitaker, penelope cruz, ashton kutcher, sigourney weaver, rachel weisz, billy bob thornton, catherine zeta-jones, benicio del toro, keanu reeves, new film, ewan mcgregor, jeremy renner, hugh grant, liam neeson, scarlett johannson, jude law, russell crowe, jodie foster, harrison ford, meryl streep, justin theroux, john travolta, christian bale, emile hirsch, adrien brody, jonah hill, nick nolte, dennis quaid, liv tyler, kate bosworth, hollywood star, amber heard, javier bardem, robert deniro, evan rachel wood, helen mirren, milla jovovich, blake lively, james franco, vince vaughn, joaquin phoenix, diane kruger, upcoming movie, robert pattinson, michael douglas, courteney cox, richard gere, daniel craig, sylvester stallone, latest movie, rachel mcadams, josh brolin, jennifer lawrence, brangelina, oscar winners, hugh jackman, zoe saldana, oscar nominee, dakota fanning, josh hartnett, annette bening, mila kunis, emma watson, david fincher, megan fox, quentin tarantino, ben stiller, a-lister, kristen stewart, charlie sheen, christoph waltz, christopher walken, michelle pfeiffer, phillip seymour hoffman, thandie newton, amanda seyfried, ethan hawke, liam hemsworth, morgan freeman, robert downey, owen wilson, olivia wilde, costars, paula patton, casey affleck, kevin costner, clooney, clooneys, andrew garfield …
- 102. Terms similar to greenish bluish, pinkish, yellowish, reddish, brownish, purplish, grayish, yellow-green, orange-yellow, yellow-brown, yellowish green, reddish brown, orange-red, pale green, whitish, reddish-brown, greenish yellow, mottled, pale yellow, greenish-brown, greenish-yellow, yellow-orange, orangish, red-brown, bluish-green, dark brown, greyish, yellowish-green, bluish-black, reddish-orange, orange-brown, yellowish-orange, yellowish-white, brownish red, pale orange, bright yellow, deep yellow, blue-green, paler, brownish-red, bluish-grey, blueish, green-brown, pinkish-brown, golden yellow, blotches, yellowish-brown, brownish-yellow, golden-yellow, pale, grayish-white, coppery, creamy yellow, greyish-white, pale gray, purple-brown, olive- green, pale brown, blackish, brownish yellow, tinge, dark purple, light yellow, red-orange, dark red, rusty brown, brownish black, purplish-red, mottling, bluish-gray, yellowish brown, greyish-green, dull red, dark green, creamy white, purple-black, yellow brown, pinkish red, greenish-blue, reddish purple, bright red, reddish-purple, grayish-green, greenish-white, pale cream, creamy-white, brownish-gray, white spots, silvery, dark grey, dark orange, purplish-black, grayish-blue, purple-blue, greenish-black, yellow spots, bluish-white, purple-red, pure white, light brown, various shades, grey-brown, pale grey, orange-pink, brownish-black, brick-red, purplish-brown, olive-brown, brown colour, speckling, pale blue, brownish gray, deep orange, grayish-brown, blue-black, darker spots, brown-red, yellow patches, gray-black, coloration, reddish color, bluish- purple, green patches, pale red, chestnut-brown, brown streaks, yellow green, lemon yellow, pinkish-red, flecks, dark reddish brown, black spots, grey-black, lemon-yellow, pinkish-white, deep red, brownish-grey, dull black, purple spots, darker green, red spots, blue-grey, splotches, grey-green, pink-purple, greenish-gray, violet-blue, silvery grey, chocolate-brown, yellowish color, cream-coloured, orange brown, small white spots, light orange, brown-grey, violaceous, dark-brown, streaked, green veins, olive brown, olive green, brown markings, gray-green, pale pink, dark blotches, light green, grey-white, dark markings, brilliant red, light violet, blackish-brown, greyish-brown, color ranges, brown-black, orange red, yellow colour, yellow color, red brown, orange markings, small black spots, veined, brick red …
- 103. Terms similar to worse even worse, far worse, very bad, horrible, terrible, awful, horrendous, bigger problem, suffer, things worse, horribly, unfortunate, better, worst, bad, complain, real problem, after all, unfortunately, no good, too, lousy, atrocious, even less, even so, very poor, far more serious, miserable, intolerable, terribly, serious problem, trouble, worrying, bothering, blame, no better, worsened, bother, worse off, dreadful, hardly, horrid, big problem, real concern, fortunately, main problem, sooner, major problem, hopeless, excuse, serious problems, way worse, complaining, horrendously, abysmal, better off, worried, inevitable, wrong, marginally, even, rid, frankly, anymore, bothered, bothers, worry, uglier, sadly, even more, worsen, severe, serious, unacceptable, badly, nasty, different story, worse problems, main reason, worst thing, far less, go away, hurt, obviously, seriously, serious trouble, hurting, gotten, anyone else, worse.it, anyway, happen, worst cases, say nothing, appalling, main concern, somehow, obvious reason, troubling, simple fact, unbearable, problematic, huge problem, worst one, exacerbated, afraid, tired, blaming, painfully, suffers, much, ironically, do anything, embarrassing, worse things, inevitably, same problems, bad problems, anything, real reason, everyone else, atrociously, unpleasant, thing, worse again, apparent reason, needlessly, ignore, seemed, horrifically, worth noting, biggest problem, real issue, even more serious, dreadfully, worsening, useless, even though, probably more, some people, pitiful, worrisome, far more, because, deplorable, point out, but, stupid, admittedly, pudgenet, worst part, less so, little improvement, grossly, make things, unnecessarily, too bad, crap, bad thing, laughable, problem, might, trying, exaggerating, pretty much, lot, doing anything, ridiculous, little reason, misguided, exact opposite, worse not better, even when, weren't, inconsequential, simple reason, expect, avoided, something wrong, counter-productive, dismal, appallingly, far more likely, ugly, almost everyone, shame, wonder why, less, polfbroekstraat, worse here, plagued, worse though, honestly, bad situation, nobody, pathetic, certainly, plain wrong, almost nothing …
- 104. Semantic Distance for Sentences I like the sushi restaurants in Palo Alto. A dromedary has a single hump. My nose is itchy! Mind the gap! The Japanese lunch place near Stanford is my favorite. Uni is actually sea urchin eggs. I wish I could eat out more often!
- 106. Demo: Translation
- 110. Virtual Assistant : Car, Appliance, Robot, IoT, VR, ...
- 112. Content Centric Siri, Cortana, Alexa, ... - Content Centric - Question - Answering - Light dialog - Context Sequence(s) - Knowledge or Actions Far from the Human communication HER, Sarah, HAL (or not ;p) - People (person) Centric - Human like dialog - Empathy & Emotion - Global Context - Concept Learning Human emotional communication People Centric
- 113. What we will learn How to to acquire large corpora and solve common NLP tasks. The nltk and gensim libraries, in Python. Vector representation for text (Embeddings). Different examples of text classification. The Deep Neural Networks that perform best on text: LSTM, GRU… Generative models.
- 114. If you remember only one thing... NLP is hard, but traditional techniques work pretty well. Nice progress since 2012, we are getting our hands on “semantic proximity”. Rapid progress on classification, translation. But no true “understanding” yet.
- 115. Take a Break!
- 116. How is this Relevant to You?
- 117. Do it with Ethic. Always!
- 118. Problem solving ?!? Imagine how to solve one of your daily problems through Deep Learning