ImmersiveAmbientModecolor: #dd9da7 (color 1)
Video Format : 22 (720p) openh264 ( https://github.com/cisco/openh264) mp4a.40.2 | 44100Hz
Audio Format: Opus - Normalized audio
PokeTubeEncryptID: d58ca02c75d0fb8a859d0744328cf0f9d4ccc59cc04f0d9f15daafe62f3ba6cb6afe4047a839a41e317dd62e5b1141c7
Proxy : eu-proxy.poketube.fun - refresh the page to change the proxy location
Date : 1716000514781 - unknown on Apple WebKit
Mystery text : andaUUQwQ3F6NG8gaSAgbG92ICB1IGV1LXByb3h5LnBva2V0dWJlLmZ1bg==
143 : true
14,809 Views • Jun 23, 2022 • Click to toggle off description
www.patreon.com/theaiepiphany
👨👩👧👦 Join our Discord community 👨👩👧👦
discord.gg/peBrCpheKE
Kicking off a series of videos where I'll be going through the actual code of many of the papers I've covered over the last few years!
In this video I do a code walkthrough of OpenAI's CLIP model from the "Learning Transferable Visual Models From Natural Language Supervision" paper.
Let me know what you'd like me to cover next!
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
✅ GitHub: github.com/openai/CLIP
✅ Paper: arxiv.org/abs/2103.00020
Learn about Byte-Pair Encoding:
✅ en.wikipedia.org/wiki/Byte_pair_encoding
✅ leimao.github.io/blog/Byte-Pair-Encoding/
✅ Video: • 1 5 Byte Pair Encoding
✅ • Subword Tokenization: Byte Pair Encoding
Unicode:
✅ dmitripavlutin.com/what-every-javascript-developer…
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
⌚️ Timetable:
00:00:00 Intro
00:02:00 High level overview: Interacting with CLIP
00:26:11 High level overview: Prompt engineering for ImageNet
00:40:25 Deep dive starts: vocabulary and byte-pair encoding
00:49:00 Vision Transformer & Text Transformer explained
01:02:00 Tokenization walkthrough
01:09:25 Encoding the image
01:15:15 Encoding the text
01:23:15 Learning a linear probe
01:27:00 Tokenization of the (brain emoji)
01:29:56 Outro
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
💰 BECOME A PATREON OF THE AI EPIPHANY ❤️
If these videos, GitHub projects, and blogs help you,
consider helping me out by supporting me on Patreon!
The AI Epiphany - www.patreon.com/theaiepiphany
One-time donation - www.paypal.com/paypalme/theaiepiphany
Huge thank you to these AI Epiphany patreons:
Eli Mahler
Kevin Stone
Petar Veličković
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
💼 LinkedIn - www.linkedin.com/in/aleksagordic/
🐦 Twitter - twitter.com/gordic_aleksa
👨👩👧👦 Discord - discord.gg/peBrCpheKE
📺 YouTube - youtube.com/c/TheAIEpiphany/
📚 Medium - gordicaleksa.medium.com/
💻 GitHub - github.com/gordicaleksa
📢 AI Newsletter - aiepiphany.substack.com/
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
#CLIP #contrastive #codewalkthrough
Views : 14,809
Genre: Education
Date of upload: Jun 23, 2022 ^^
Rating : 4.755 (26/398 LTDR)
RYD date created : 2024-05-11T17:15:11.101928Z
See in json
YouTube Comments - 39 Comments
Top Comments of this video!! :3
someone give this man a nobel prize
8 |
This is great! With all the new fancy models I kind off felt left behind, but this is surely going to help me to learn how these models work under the hood. Thanks, and make more like this!
6 |
Dude, I LOVE this format! Casual code AND math explanation, how awesome?!?!?! <3
|
Great explainer Aleska! This is going to be so useful for many. Thanks for sharing. Mike
2 |
Great video! Loving these series
1 |
amazing job! Thank you!!
|
It has been a while man!
Always happy to watch your videos!
1 |
Great stuff as always! A not necessarily related question (but came to my mind after seeing you using PyTorch here): Do you have the freedom of what framework to use at work, or is DM fixed on their JAX ecosystem? Looking forward to the next episode in the series 🥳
1 |
thanks, these code walkthroughs are super helpful. keep doing more such videos.
1 |
1:20:25 - Its the temperature parameter (usually a tuned hyperparameter, but a learnt parameter in this case)
|
I'm trying to contribute more to Disco Diffusion.
This video is fantastic. Thank you for putting it together.
1 |
this vedio is so gorgeous! and it helps me a lot! Thank you so much!
|
Can we have code implementation of neural sheaf diffusion, ricci flows and hyperbolic graph convolution. I was liking that flow of works…. So maybe some code demos will be very helpful to see how i can implement them
2 |
Good content! I actually want to work on something similar🤣 When it comes to feedback, I would suggest more high-level overview of functions(maybe like a list or visualisation) and overall model structure before you dive deep into the explanation of nitty-gritty details of the code. Nevertheless, great job bud!
2 |
great explanation!
|
1:20:00 is there a proper explanation of why there is a logit scale factor when calculating similarity? Thanks.
|
great explanation. can you do a series on NeRF
|
watch with 1.5x and in some text encoding moments think this guy is little fast,(lol) but it was great. go ahead and wait new content like this.
|
What does clip actually encode into the length? When normalizing, don't you lose some information?
|
@TheAIEpiphany
1 year ago
Super hyped about this one! Kicking off a machine learning coding series! I'll be walking through the code behind many of the papers I've covered over the last few years - starting with OpenAI's CLIP! Do let me know how you find this one - feedback is very much welcome! Is the code too tiny? Too many details? You love/hate the format? Whatever do let me know!
10 |