Vector Databases & AI: Fact Check and Technical Overview

Lance you ready yeah cool welcome to hack Sunday I think I'm pretty sure I know everyone here in the

room uh thanks for joining us today we have uh gav here we recording to trick beautiful um so I will let gav introduce

himself but basically on our chat there they were talking about Vector databases and Stanley's tried to explain Vector

databases to me and I said grov I have no idea what you're talking about so will you please put together a

presentation for us to teach me because I still don't get what these are so uh grav have about a quick about yourself

for anybody who doesn't know here yeah so I'm just a senior software engineer uh working at a insurance tech company

um I have had uh some um some chance to research into computer vision algorithms and um lately I have been into

generative Ai and that's where I started dabbling into uh Vector databases and uh um I also got a chance to write a

chapter on Vector databases in one of the Playbook so that's what uh um that's that's what triggered Jerry to uh invite

me thank you Jerry uh to give this talk and let's get into it beautiful so let's start with uh what is

database and I I don't want to make this talk like uh somewhere I'm just talking giving a lecture College lecture so uh I

would like uh this talk to be like as interactive as possible and we do have two wireless microphones right there

mics right here anyone with questions pass them around

know it's not live streamed who asking wait who is asking I want to know

Josh what's Josh show me oh show me oh cool huh maybe next time we will um yeah cool so as we all know database

is a structured collection of information and it's it's like a software that people use used for in uh

information storage and retrieval right but over the time these databases started uh evolving into into different

types and we now have different types of databases so now we have document databases we have relational databases

these are the most commonly used databases everywhere then we started into analytical databases which is like

uh uh analysis that Stanley does uh on his um on is genon mix and everything then we have these graph databases which

are really good for relational um U for storing relationships and Facebook is is the greatest example for uh use of graph

uh databases and then people started thinking why only save databases in in rowwise right C can we store them and

retrieve them in column wise so we started with column n databases and then finally as um as everything is is a key

value pair so a key um connects to a value and there we have like key value pair databases before you go too far

with that so you know again maybe I'm the dumbest person in the room and also some rudimentary questions here but I am

already getting lost in this so you just said that Facebook is a graph database Facebook uses graph database to store

the relationships that we have with our friends how many people knew that okay all right so I'm not alone

over here but all right and if you um um don't don't mind me just jumping in for one second sure um is I love to think

about the contrast between relational or tabular databases and graph databases because I feel like we all naturally

think in graphs like I see GV I know we're connected by a friendship and the same with Jerry and and you almost think

that there's a spiderweb of uh relationships that encodes the social structure of this community here um and

and that's what our brains do is they're like you know set to think about the relationships and the relationships

between people and so I always have to feel like it's almost like we contort ourselves a little bit to think in

tables yeah yeah so uh and and again I I know uh Stanley you would be you would be dabbing into graph databases pretty

soon so um that that's a I I am like tickled pink because I'm I'm actually pushing a petabyte of data

into a graph datab starting Monday so Jerry I'm so excited that this talk is happening today bro do dope now okay

does everyone else already know what analytical databases are because that that's also a new one to me I do not so

analytical analytical databases are for example Amazon dread ship now you have so relational databases oh sorry yeah

first time so relation relational databases uh are usually you can just store a limited

amount of data now if you want to do some um that's that good yeah hello yeah so if you want to do some historical

analysis for example on stock market how was the stock market in 1930s so you would like to store all the transactions

into an analytical database so that it can it can predict the patterns that are happening so and again it's very hard to

do that in a rowwise like if each row is stored um in a uh in a relational database it it would take months to to

process your query but when you store it at like in a analytical database which us as a columnar storage it becomes very

easy and it's efficient in running those analytical type of queries I think I see a possible

followup presentation coming yeah like every topic is a presentation in itself and I'm just

introducing um these things to to get further I'll let you get to Vector ones I won't keep holding you up I already

got tons of questions what what we might say though is that each of these databases might you might think of it is

they have a certain type of question that they exist to help answer right and and then and and we'll we'll dab into uh

like what are the use cases of these databases as well uh in our upcoming slides yes I do have a question over

there uh I was to say one of the other benefits of cube so that you kind of preprocess is that you can pre-process

the data right so that you can get like daily weekly monthly annual summaries for like sales quarters so if that

information exists for each individual order in like a sales data warehouse you can pre-calculate those on some

frequency so that when you're querying it it just happens instantly so it also empowers things like ad hoc bi where

like if you want to like slice and dice data in different ways you're not waiting for like the relational data

warehouse to constantly sum up average minmax and run all those aggregate queries across the RO level data

yeah so now we talked about like mostly different types of databases so now

databases can be categorized into three categories SQL which are like relational databases no SQL which are these column

based key value graph document and there is something called as object databases as well so so like I don't know many

people might know um so for column databases uh the good examples would be Amazon red shift which you use for uh

business analytical data business intelligence Cassandra Maria DB and then for key value pairs there are red uh

redis mcash couch base for document databases uh you can think about mongod DB elastic search Dynamo DB and for

object databases uh we have our pretty old friend Ms AIS and uh object DB is is a new one that's a Java based one and

then uh for for graph databases um I didn't find many examples but Amazon is coming with a new database called

Neptune neo4j bro and neo4j sorry forgive me sure and Ne 4G so um would also toss out Janice graph is

the kind of typical open source solution but neo4j is just so good that's what I'm

using right get that mic in your mouth for me look not on oh that's not

onu wa language to query databases that are traditionally nonse such as a gra datab

so and where that comes into play is fql which is Facebook query language a query like language that is used

ons yeah many other databases also have a SQL like language because it's great SQL is is

useful did you get it on thank you oh y well yeah yeah thank you

Mike so uh as I said like we'll go through different use cases so these are like some of the use cases relational

database you basically use it for payment system booking system um even storing your daily transactions then as

uh as a friend over here said like uh column storage databases are used for business intelligence then you have have

Object Store which stores like files audios and pictures then graph databases as uh Stanley said like it is used for

social relations and uh um anti-money laundering that's a um uh that's a nice use case for it then time series

database you have like stock market data trading platform Stock Exchange and uh in memory store is like cash or like

flash sale kind of database uh Usage Now Let's uh jump into Vector database so what is Vector database um

so I would like to think now we thought uh databases to be rows to be columns to be graph and now someone said let's

think about Vector uh this data storage in in terms of like 3D space so each data is a is a point in 3D

space so that's the that's the Crux of a vector database so basically Vector databases stores the data as

mathematical representations and these points can be termed as Vector embeddings basically uh

all so as you can see in this example all the animals wolf dog cat you can cluster them into a certain space and

the fruits which are apple banana grapes whatever um they are clustered into certain space and then you can make a

semantic relationship between those points and saying that these these samples belong to this category yes yeah

are vector databases traditionally limited to only three dimensions no it's it's just

a it's it's just a represent ation uh a way of representing the logical point of it they could be end Dimension right so

you can have your data in end Dimensions but just to just for the comprehension of the human mind since we

live in 3D space and we understand 3D space really well uh that's how it is represented um but thankfully the the

mathematics of uh of geometry work in whatever Dimensions so we can do things like take code signs and you know apply

other kinds of mathematics arbitrarily that's exactly um actually that's the next slide uh next to next slide but

let's dive into how do these Vector database actually work like how how how are we using this for quering for using

that as a traditional database which is like information retrieval and storage so as I said Vector databases do not

store your data as is they they store the representation of that data they store the point location in the 3D let

not me uh let me correct myself over there not not say 3D but n dimensional space um in the computer so that's

called Vector representation and then they use something called as finding nearest neighbor and we'll go through

those algorithms in the in the later part of the presentation and then even your query is converted into a Vector

representation and then we try to do a match and that's what the result would come out yes not necessarily jumping

ahead just seeing the correlation between that and machine learning where you take your data and remap it into

some sort of vector that you're actually learning over right training over okay yeah it's uh um kind of interesting I've

worked on uh graph data driven Technologies for for quite a while um I think when I started it was estimated

that about about 1% of the world's databases were of uh graph type um and you know throughout my whole career

we've been kind of watching that steadily grow um and it was predicted but by the end of the decade we might be

up to 15 or 20% of all the databases in the world would be graph databases um it it's also kind of fun to remember that

the if if you think of the market of all databases in the world like as an economy of itself we're talking about a

big country like we're talking about one of the most important economically impactful Technologies and it was

already going through this huge change where we were storing the social data in graphs it was already a major shift in

the industry llms are putting that on steroids and I I'm sure that'll come up in the talk but it's sort of like

everyone's going to be learning about graph databases and talking about it within the next year it's kind of

amazing um it it was actually leaked uh in a talk a couple months ago that that's how open AI is fine-tuning chat

GPT as they have a big graph database right so uh let's let's get into a little bit of mathematics so now you say

you have this n dimensional grid and now on this grid you need to find two um two points and

their distance because we want to find their nearest neighbor right so these are some of the uh some of the

techniques uh that that computer scientists are using so for example dot product it is uh it's it's it's a metric

that measur the similar ity between two vectors by uh evaluating their sum of the coordinate points right as you can

uh see over here uh the cosine distance is the uh similarity between the two vectors by assessing the cosine angle

between them and uh comparing with their magnitudes um then this is the most famous um ukian Square we we all know

that we can uh identify the distance between two points with this uh formula and um I I like to think Manhattan just

uh just like in Manhattan New York everything is a grid and to get from point A to point B you cannot make a

diagonal um you cannot take a diagonal route so so that's how the Manhattan distances are also known as a taxi cab

yeah the taxi cab metric cuz yeah you can only go up down left right there's snow um kov do you have a a a manom king

queen slide in there or no I I don't have that no problem problem if you want to go ahead and uh give us uh some more

context on that that um 100% real quick and forgive me it's just that the um I I feel like seeing the words floating

around in the space is very compelling um I almost remember that we have an expression reaching for a word you know

when you're trying to remember a word and you can feel that it's floating out there somewhere I I think our brains use

geometric embeddings of words to help us keep track of what we're saying right um but then one of the things that's really

interesting about this is the way that the words relate to each other geometrically ends up reflecting their

linguistic meaning so for example if a word differs from another word by gender like say man versus woman or King versus

queen the distance between those two words will always be the same and so in a sense like when you embed our language

in a geometric space like that the things our language says show up in the geometry of the space so for example um

all of our language is really split up by gender and and you can see that based on where the different words float in

the space it's really kind of an incredible thing yeah awesome love it so next like uh what

sort of searches can we do on Vector databases so let's so these searches can be converted into two different types

one is like traditional search this is like the SQL that you use right and then you can see over here you are looking

for like like cat like dog and it's it's like you are searching for that specific keyword in your database whereas in

semantic search as Stanley explained the it's it's looking for the overall geometry and trying to see uh

where that concept that context relies in the 3D space and tries to give you um to give you that particular answer do

you think we could make a better language for searching this cuz on the right that's just a bit of code and on

the left you have a a language a formal language to to query it do you think do you think we can do that uh yeah so

people are uh trying to make it uh english- like uh and and I think that's um that's how like programming languages

are developed right first you have first you had assembly where everything was machine instruction given and then we

started with english- like language like kobal basic and then we came to cc++ Java and now python is just like writing

English language right uh so a lot of these semantic searches are also find way microphone

please thanks sorry uh a lot of these semantic search Primitives are also finding their way into traditionally

relational databases like postgress has a an extension called PG vector and there's two or three three other

extensions that are coming out at this point that basically do the the query on the right um but inside a a SQL so

exactly beautiful so there for that that's awesome are you saying that it will take the semantic search and then

create an SQL search based off that yeah so with with PG Vector it allows you to have additional uh

functions inside your uh inside your select statements and inside your where conditions so you're able to do

basically like a multiactor search so if you wanted to like search all of a single users preferences you'd be able

to use traditional SQL to filter on user ID equals you know 1 two 3 4 we would be doing that today as part of our demo so

we would be searching for some images of of like famous uh celebrities and uh you would be searching it based on based on

their file but it will give you um it will give you the results based on based on their features based on their

facial features so we would be using uh a facial recognition basically um as part of our demo cool and it would be a

SQL like query that you will run but uh you would get an image as the result um grain of salt because I'm a big post-ess

guy but used PG V in production recently and it was very very nice awesome also have a quick proposal I think when

talking to normies who don't understand understand AI replace semantic search with like Vibe search you know we're

just search them for that General Vibe um that is um totally true 100% um another thing to mention too is just I

feel like the Spectre of llms hang over all of this right um I was working with a thousand row data set called spider

that can train any open source llm to convert from uh text to SQL queries and it it's like

99.5% accurate it's very impressive with with almost any open source llm wow yeah so there without any further

Ado let's jump into the first one the first demo um Jer is this on the website or do you want to share this on the

telegram or oh we could take technically stream on a telegram in the future I kind of

wanted to test that yeah the screen is a little washed out I think it's mostly from the

Skylight but you think maybe do we have these for I I can turn the spotlights down a

well so in yeah so in this particular demo what we will do we'll uh we'll try to create a vector store from scratch

and what we would be doing we'll be doing some semantic search uh we'll be getting the embeddings from open aai and

that's why uh I won't be sharing my open a API token with you guys and uh we would do some metadata

filtering so so as as part of it uh you can download like any any sample documents um over here I've just uh

downloaded this archive um and you put the link to this in telegram yeah uh yeah Okay cool so any anyone that wants

to follow La just jump on telegram links in there yeah the link is already on the telegram and I want you guys to um run

through it with me um so then we'll we'll do some python installations uh dependencies and then finally um we'll

able to uh yeah of course this is um this is loading the PDF data and then we'll pass it into nodes and here's the

here's the cool thing where we'll get the embeddings from openi since this is a text search we'll we'll use the

existing embedding uh we don't need to go through creation of embeddings uh as part of this uh this demo today um yeah

over here uh you guys will put your open API key embeddings are the vector representation

of yes embeddings are the vector representation of the data itself so open aai was working on this uh text

search for many years and they they created all this uh context for English language basically they have created uh

representation of a n-dimensional space for English language and that's what we call as Vector embeddings for English

language and so they use GP yes so it's again the embeddings are

trained on different data right so GPT is trained on on a different set of data what uh we would be using today is text

Ada 002 embeddings when it says their llama Hub is is that part of like o

Lama I keep saying it says l i don't I think that's the only one that doesn't say it every other

one if you go up or down either of them will say it okay uh llama 2 is a popular base

model yeah are youting the EMB with open AI I'm just using their their embeddings I'm I'm not generating anything oh very

cool yeah so I'm just using guys I'm sorry you're G have to explain that one what what are embeddings and why are we

using where the words get to float in space okay the embeddings of the word in the space they embedded in the space and

you said you're using open AIS do they already have embeddings that we can use yes so so you lost me so so when you

think about this model right llm basically what you're looking at is the uh is the embedding is the vector

representation of of the context say suppose in this case it's the English language so you know this is noun this

is adverb this is uh this is an adjective and you make sense of it right as as a human so similarly these models

were trained and when you train them they create the data representation which is like the vector representation

as Stanley explained these are nothing but points which are floating in space right and then when uh when you start

like doing the query what the what the mathema mathematics behind it is like based on your query it tries to find um

what's the what's the nearest Point what's the nearest word that that relates to this thing and that's what

the semantic search is all about the nearest thing that relates that word in in general or in terms of

your search in terms of your search in terms of search okay so that's what they call is it rearranging this database

like live it's or it's just it's just calculating I guess it's just calculating at that point it's it's not

rear I like the picture of your rearranging happens when you train or when you do fine tuning that's when the

re um at the same time though the process of creating the embeddings does does have some things like this where it

sort of like it tries to fit some of the words in then it measures how close those are to being efficient and then it

puts new words in and so it is an AI process to even generate the embeddings though

right all right does everybody understand that it's like a game basically right

I wonder is is it possible we do a quick Google image search because this is actually the canonical you want to you

want to search and pop it on the the telegram and then he'll just pull it up all right so while while you're pulling

that up I'll I'll try with with just words um so you know we as humans we think of

Concepts in who knows how many dimensions right so like a human can be uh any number of traits they can be a

friend they can be an enemy they can be royalty they can be a criminal they can be tall they can be alive they can be

dead like hundreds of different dimensions that we represent in our minds for a human a plant is also you

know potentially a living creature right a rug is none of that right so we every single thing that we learn about we have

all these weird attributes in our mind so Vector embeddings are a way for computers to try to

represent up to however many dimensions are trained for that model so for example open AI Vector embeddings I

believe have 1,536 Dimensions right with the text Ada I believe it's 768 dimensions and every

single one of these Dimensions is a value between minus one and one it's a floating point value so in order to

generate an embedding function you take some Corpus of text for example all of Wikipedia or like every news article

that's ever been published and what you do is you kind of walk through each of those uh bodies of text and you try to

like remove one of the words in a phrase and then you try and guess what is that word in that phrase and the better you

can guess what word fits into that phrase uh the better you can train how these 768 floating Point numbers kind of

relate to each other in the space so if there are things that could possibly fit in like my favorite animal is a cat dog

right almost no one says giraffe right so giraffe is GNA be farther away from cat and dog than dog is from cat or cat

is from dog but you do that for every single phrase across this Corpus like every single phrase in Wikipedia every

single comment thread in Reddit and this is how you can have these various different ways to model how humans

represent different topics so the canonical example of uh king and queen yeah so the canonical example of

king and queen is that you can use the representation of King as a mathematical Vector which is these 768 numbers

you can subtract out the vector for man and then add in the vector for woman and these are just really simple

mathematical operations subtraction addition and then you look inside this multi-dimensional space and that result

is going to be the closest to the term Queen because you subtracted the essence of a man you added the essence of a

woman from King and then you search in this space of you know whatever millions of words exist in this database and the

closest one you're going to find is Queen yeah so yeah uh woman plus royalty equals Queen we get to do algebra with

words and hopefully we'll just get this this picture up that'll just kind of hopefully make it a little bit more

concrete but I just feel like it's so fun because um you know there's so many things about the language we speak we're

not consciously aware of like the way that words have different resonances with different genders or different

topics it's it's just really amazing because these things make All That quantitatively explicit there you

go cool and forgive me for for wanting to just get this up there real quick G Rob

I just remember the first time I saw this it was one of those big head exploding moments for me as a

student yeah and I think to get back to the the original answer there's a difference between an embedding and an

embedding function so an embedding function can take something like a word or a sentence

or an image and then return that embedding so when we say that open open AI has all these embeddings um sure but

really what you're providing them is you're providing them some input then they vectorize it for you and they

return the vector embedding which is this big group of numbers for whatever you're giving

them so you give them uh you know a sentence they return a sent sentence Vector you give them a word they could

potentially return a word Vector you give them an image they could return an image

Vector but that function is I believe proprietary to them but there are open- Source versions of these that you can

use for free that you can basically chunk up whatever your own documents are and uh and get a set of vectors uh just

by using the the compute on your MacBook I did also like the example you used was people and their pets because

on our Twitter spaces we we talked about artists and how they have the strangest pets so now now I'm thinking okay well

maybe if they're an artist maybe they would be closer to the [Laughter]

giraff nice um Sten did you want to tell us about this oh yeah okay so this is just an expression of what we're talking

about so you know again the idea is we want words to exist in the space so that their geometric relationship like how

they near each other um reflects the linguistic uh information about them so so again on the left there you can see

the M male female difference so we have a man and and women woman up there and then we have king and queen down there

and so this Vector that goes down into the left that connects man and King and this one that goes from woman to Queen

um in a sense that Dimension that direction encodes the concept of gender as it pertains to the um the language

um similarly in the middle there you can see verb tense becomes a dimension right so walked relates to walking swim

relates to swimming every verb would be connected to its uh uh tense through that same direction um similarly on the

right there uh the relationship between uh a country and its capital is is structured right there um so again it's

just really interesting because it's like it doesn't need to be told this it learns this by looking through all of

the text and seeing that there is this pattern between when man occurs when King occurs when women occurs when Queen

occurs um so it's just incredibly cool it's like just you know taking these Concepts that we learn from our language

and again making them quantitatively explicit I also like how swimming and walking are slightly further away than

walked and swam so it's blowing my mind it's say well are they that somehow different than the past

tense and and the answer is like probably yes like there are these little differences and it's not perfect but

then you are able to do computation with it and it's uh quite fun yeah and explicitly on the the capitals one it is

about whatever is in that body of texts that it was trained on so when I ran these when I was playing with PG Vector

for postgress uh Beijing and Shanghai were like almost like equally relevant to

each other so even though it could get like the capital of the US the capital of uh you know France the capital of

Canada it would sometimes say Shanghai instead of Beijing for China right so it's not like it knows that you're

looking for Capital it it just says like this is something that is like semantically similar with whatever

mathematical operation you run which means that it's not like a fully it's not like a foolproof Knowledge Graph

right it it is like something that helps find things that are potentially relevant but it might be relevant in a

dimension that you're not expecting yeah and and that's what I uh I also said like in my LinkedIn post is like these

are all probabilistic algorithm there is a probability that Shanghai is a capital of China we know as a fact it's not it's

Beijing but for the computer since those cities are very similar in um economically their population and stuff

like that so it can think that it is one of the capital um and and 100% it B it depends on the data that you're using to

train um I once had a role where I generated a lot of vector embeddings for for Google in many different areas and

so actually the the degree to which gender existed as a dimension is one of the first things we checked with the new

series of embeddings and you know if we were using social data it would always be there but sometimes we would be um

creating embeddings from like medical documents or scientific papers and then they would have very different things

captured by the model so it it's just an incredible tool though and as a person who had like linguistic training in

school it's it's just a amazing what you know the the questions we used to ask ourselves just over a beer in musing

that we can now answer kind of explicitly so so as part of this demo basically what we have done we have used

this embedding and uh uh used like on on that particular PDF document and finally um we were able to ask a query string

saying can you tell me key concept about generative Ai and this is the answer that it came up

with so I don't know if it is true or not but but yeah this is this is just based on on that particular PDF and the

English language embeddings that are given from open AI right um and I I I do see um like sometime

back on uh on our um on our telegram Channel um um there was a question can we build a rack system using these

Vector databases and my answer was yes and this is like another example that uh that basically you can create a rack

system which you can use for like fine-tuning uh your llms um can be built using uh the vector databases sorry a

rack system so uh retrieval augmented gener ation what um what these are is like um you you can you can train your

llm basically using these uh these rack systems you can uh you can feed them different data and they'll try to create

their own idence for for that particular data is that I I'm scared to even ask this but is that related to

Lura um no not specifically so um luras are a an approach to fine-tuning a model and then rag is different because rag is

almost like you give it a long-term memory through one of these Vector databases and the fact that the database

has this linguistic uh layer allows the llm to actually find stuff in it so as an example uh we created a rag system

for our friend John's Medical Records he is a chronic illness patient who you know has been seeing doctor after doctor

after doctor for eight years getting lots of testing done he has a 4 terabyte solid state hard drive he has to bring

around to all his doctor's appointments and then he has enormous trouble just getting like the right information

together for like an insurance request so with the rag system you could feed it the embeddings generated from his

medical records and then say show me every genetic sequencing test that's been done and then it would would find a

mistem that's a rag system right um Laura is fine-tuning the intelligence of the model which is like a different way

to change the way the model performs and actually for state-of-the-art stuff people doing both so it's kind of fun

right a lot of compies use it for their microphone my friend a lot of companies use it for

their um their own internal policies so they can put all their policies on one thing and then have somebody go through

and ask okay well how do we do this particular thing and you know have it all sitting right there for it yeah at

my company I built uh a rack system basically to um to understand where our test were failing so that we could

automate our testing capabilities so so that like every time a test fail it goes through the rack systems and say hey I

have have I ever encountered this particular error and then if it is there it knows how to solve that so um we are

using we are also using um something called as uh uh dop um to to basically train our rack

system to give like specific answers because sometimes you get a 500 error and it could be um uh it could be a bad

server or the kubernetes SP got down and and that's why you got an error but it will try to change the code which is uh

which is a false positive at that point uh I do have a question yeah yeah hopefully I'm not going too much Back to

Basics but the embeddings is that what the vector database stores so like you know how like ultimately like for SQL

you're going to be storing some sort of like floats and strings and stuff like that I'm storing embeddings that are

generated through these systems and then that is then retrieved and then the the vector database job is to maybe do some

of these algorithms to find nearest things where you query off of am I understanding that where the functions

and Authority is lined with this yes yes so in the vector database and and and that's the that's the beauty of it right

now say suppose you create a traditional database a document store which stores like all your images now that dat data

would get exploded it would go into petabytes if you start storing like each image with with its metadata right but

now if you are storing just some mathematical pointers coordinates in space that requires like maybe a bite of

data and now you can store like I don't know six 16 million images uh that way so uh basically you are just storing uh

you just storing the coordinates in the 3D space or end dimensional space got it so the the the the the struggle is the

compute side because you got to basically determine those derivatives and those uh context of the data itself

so like if you store let's say 10,000 images of a dog it's completely tensive to figure out what those like meta

things are but once it's there it's very small and you can store it into a database got it so so it's very compute

heavy for the gain of very storage light and retrievable light right cool makes sense thank you another question yeah um

I was uh wanted to clarify the distinction between Rag and fine tuning again I wanted a little bit more clarity

on that so is uh with a rag system you're adding embeddings to the existing embedding database is that is that true

or is that something else or and then with fine-tuning are you just

updating how the query responds to the existing embedding so I'm just curious on the distinction between a rag and a

and aine tuning fine tun yeah U as um as Stanley explained fine-tuning is mostly like um changing how the embedding would

be generated by the llm itself right so you are you are adding um your adding more context to it

like in a uh in a heavy way or or a 50,000 ft uh so you're recreating the embeddings yes you are recreating the

embeddings whereas in a rag you are adding to the existing embedding say suppose now you have embedding for

English English text right but you want to make your llm very specific to I don't know for lawyers right now um now

you will need to add all that Legal Information the legal language uh semantics to your uh to your embeddings

and then at that time using a rag sort of system helps so if you're building a rag on top of like say open AI you're

taking their existing uh embeddings and then you're taking your knowledge Corpus and you're replacing the some of the

embeddings in the existing embeddings uh with the rag or you're just adding a whole new set of embeddings I would say

it's it's more better to be additive at that point of time because if you are if you are creating so so basically um the

whole point comes out to be say suppose you use GPT Asis right yeah if if it doesn't understand your speciality if it

doesn't understand your context as as a lawyer for example then it will it will give you like a hallucinated answer

right alog together so if you want a specific answer which is which is factual based on the documents that you

have given then rag systems are really good at it right right and they they basically just added in yeah yeah

so referring to Thea low rank adaptation it's it apparently we can combine that with other types of fine-tuning when do

you personally and professionally decide if you're going to use high rank adaptation or any other sorts of

fine-tuning based on you know circumstance substantial you know whatever it is you're working on you

know um forgive me I just this is one of my favorite questions right now y'all um and no perfect answer yet um we're still

understanding what you can do with different finetuning methods and different approaches to constructing

data sets and we're finding out that there's more and more you can do we're getting more and more surprised at you

know how much you can teach these things if you do it right um Laura is really interesting um so as gurov said uh rag

is essentially taking an existing llm and giving it like a memory like a longer term memory it can access this

rag system pull up information based on what it's looking for fine-tuning is like you're actually like scrambling the

neurons well hopefully not scrambling but you're actually going into the neurons that store the intelligence of

the llm and kind of restarting the training and tweaking them a little bit um but to do the full training we

remember that these things take 10,000 gpus working for 9 months so you can't actually retune the entire network that

contains potentially billions of parameters you pick a subset of parameters to tune and yeah there's

quite a bit of complexity into what performance with what data you can get out of um tuning how many parameters

Laura is really interesting though and the punchline of what I'm saying is start with Laura and if Laura works

that's great because it's it's so much easier than any other method and then what Laura does that's really special is

it doesn't change existing neurons it adds an extra layer or two just on the end of the network so it's almost like

giving uh an interpreter to your llm that kind of takes the output and then re reframes it without without touching

the core intelligence of the model right all right uh did did everyone try uh doing the demo or should we go to the

next one it's on the collab notebook we can jump to the next one we can play with that more later yeah I was

trying to get on my phone but it doesn't seem to work or at least I didn't get that for

yet put the second one screen with this one

um this is uh uh this is the adopted demo by by single store um single store is uh just another Vector database

provider um um like pine cone or um Q dot or um any of those and what they tried to do is they tried to uh use this

face net neural network um and they were able to build their single store database with like 16 million images

records and uh today uh as part of this demo what what I did uh is basically I use their databas database and since um

I don't want to spend more money so I just use 7,000 of them and uh let me go down and again over here uh I have given

Specific Instructions for everyone um to um to create like an account uh on single

store and how to use their um uh their secret key basically uh so you should take it from the connection URL and from

there you need to copy the GWT token um here's mine and it would be expired by now so uh that's what you get right um

so I did a DOT product so as as I said that we'll be using using SQL and um just

to uh just to help us with Vector databases calculation single store has actually created a function that takes

the dot dot product of uh of the vectors for you so that's what we would be using and um as as you had you had asked like

how do we use SQL because SQL gives you um not like like but you can have like a specific uh

specific value to be found right so a similar thing we we would be doing over here as you can see um I was looking for

Adam Sandler um and uh uh I was looking for the image number three uh 003 and the vector uh value got I got was one so

that that's a perfect match but not only that it also found some other points which are close to it and this is the

probability that you can see for those vectors and Ian Thorp what was that part yeah kind of bked out on that one huh

yeah and over here interesting as you can see I tried to search for Alec Baldwin but I got Daniel commo as well

and Vladimir mcar um yeah so I mean they do look kind of similar it's kind of interesting they

do look similar and uh yeah I I want you guys to try this out and U there is a there is a list of uh I

don't but is it just looking by the file name or is it yeah it's oh but then once it finds that then it says okay what

else relates to this and then those other pictures do kind of actually look like him wow exactly that's impressive

that's what it is doing that's very interesting and um you know going back to Stanley was

saying how that's how we think too you know when you say something oh yeah that person looks like so and so exactly um

it's it's so interesting I I'm having such a blast with this talk um and I want to just make a note that I'd love

to come back after to a adversarial training I don't know if you guys have seen this but so like someone

actually fig so so it's like I'll save it for the end but actually someone hacked the brain's computer vision

mechanism using this technology mhm and they can do something that's like borderline mind

control yeah it's uh again that's um coming back to the point I said this is again probabilistic right it's it's not

it's not based on number Theory it's it's more on the probability so it is trying to find the probability like

these are the features that match this particular picture and um uh underlining this particular demo Mo uh we are using

a open- source uh uh neural network called face net which converts these these pictures into some facial features

and we store Vector embeddings of those facial features so yeah um I would I would love if some

someone wants to try and uh um go through it um and maybe we can also showcase um their favorite celebrity um

let me see so you just set the name in there and then I'll do that no actually here

is is the drop down and on the drop down you can you can find like those celebrities that we have inserted in the

database oh you've already selected them in there yeah I I've already selected Alin over here but can we change it to

Obama um I don't know that try give me Obama he's better than Baldwin Baldwin's facing

trial that whole thing is such a such AER there was some some other celebrity got hit with something yesterday got

some email was it him yesterday yeah it was in yesterday I don't think he'll get it I

don't free all the bald ones is there any chance anyone could send me the collab link on Discord I

just don't have Telegraph on my computer it's no problem that why don't you get telegram on there

he's onam well it takes like 30 seconds you know I should I should I should this computer is just my one little Lake of

Sanity in my life and so I just have an aversion to to putting too much on it but you're right especially for the

importance of this I needam you can mute it too yeah um oh thank you gra this notebook looks

amazing and I'm G to run it um all right did we kind of cover most of the stuff there now we'll just

kind of break off and everyone try it yeah try that um actually I was uh I was trying to

change uh change for um well while you do that I'll do I'll do a couple closeout messages because uh you know

for anyone who doesn't know we are here every Sunday will be back next Sunday sounds like Stanley's going to teach us

about mind control next Sunday it's kind of interesting I anyone see Bean's shoes today not yet beg is

rocking the coolest kicks I've seen in several months and they look like something interesting they look like um

an what's called an ad adversarial computer vision patch oo so it's like there's these little patches you can get

it looks like a little circle it's like a sticker and it has kind of like random looking static on it and if you have a

computer vision system pointed at you that's like actively saying like oh that's a human I'm seeing a human you

put this sticker on and it thinks you're a dog and so basically the pixels in this

sticker have been designed by a different AI to glitch out the first Ai and then the way it does it is it sort

of like measures it against the kind of system that gorov is showing so it kind of goes like Oh does this look close to

this person if it does I'm going to change some pixels and then see how it does and then that

system of changing the pixels can actually like make it so that a human couldn't even tell the difference

between the two images but the computer couldn't classify it at all very

interesting did it work yeah I'm just putting you got me Obama I don't know if I have Obama over

there but I'll I'll give you like the list of 7000

add all right well I think we'll close this out for now and we can keep on

tinkering well thank you everyone for your attention but I'm not done with my doc

yet you're not done there's more yeah we have to go through uh some some algorithms and also compare them oh we

good we're already six minutes late my friend right um uh let me let me just uh go it through pretty quickly um while

while we are looking at look at I I think too that this is a talk

that needs a sequel yeah I was going to say may maybe should we break here and you want to continue this next week I'm

I'm cool continue this next week sure okay um GV if you were interested I would even be so happy to kind of that

sounds great add a little to the mix and then maybe continue because this feeds like so well into this adversarial

example sure that I think could be really fun cool um oh my gosh I had like no idea we were going to be talking

databases today this makes my Sunday I I love databases I I started with access databases Microsoft Access databases

yeah we we did dabble into that I I used I used to store images and access databases so if you want to access my

images you had to go through my security protocols um so any if anyone is interested in learning sequel I just

posted de hack Sunday Four sequel games including a murder mystery game is an excellent way to learn Sequel and

absolutely chat GPT and Bard can help you here um yeah SQL is more powerful than ever and if you know a bit about

its history it was developed by the CIA as a secret tool to access the world's information this is real Larry Ellison

is xcia and he adopted SQL and built a billion dooll company so he's one of the richest men in the entire world off of

CIA technology um and SQL right now Oracle is um postest is amazing don't use Larry Ellison's product I'm

not endorsing him um uh where was I going with that oh yeah learn sequel games yeah it's powerful and it's only

getting better awesome thank you much it's uh it's really incredible how like SQL is

just one of the core Technologies of the whole internet Revolution it's maybe changed our lives more than anything

else except for electricity or fire oh yeah right up there yeah let's break off we'll break for next

week and we'll do a part two maybe next week we'll leave them live stream one more question all right one more sorry

um just got a curiosity prior to all this Vector databases how did they train stuff prior I'm kind of curious on the

like what the context on why this was the natural solution that people decided to go to because like in a traditional

machine learning model I'm just doing weights which could just be in a traditional SQL database and you could

Fetch and then run your model off of so I'm kind of curious on where did Vector databases start to become like uh the

Necessary Technology to continue with machine learning so as far as I know Vector databases started to pop up when

people started thinking in terms of like context right where you have to search C certain data for certain context and

that's where people started thinking oh instead of saving um uh saving like the

the whole data and then quering the the complete database why can't we why can't we imagine as a um as Stanley said like

a like a globe of words and then uh try to Cluster them also it comes from a very uh important algorithm that is used

in neural networks it's called K means clustering and K means clustering has been there for I think so since 1970s as

a classifier so it's it's it's not new it's it's a different vision of the same old technology that we are now applying

add one more thator to having a formal database prior to having a formal database if you were to look on GitHub

to solve these problems you would find individual libraries that provided this tool so for example there are plenty of

K means libraries there are also tree structures that were being used to do vector-based lookups so there's a try uh

so that is a particular type of tree that is very useful in using P hases to do reverse image lookup so if you were

for example to look up how do you do a reverse image lookup or how do you do a SoundHound people were writing blog

posts they were using these data structures in memory so that then that is basically prior to a database that's

what you use use a more primitive data structure um a database what that this does is it allows us to to be reusable

to be packageable to to be you know to have this amazing power at your fingertips without necessarily

understanding how it's being done under the hood um on that note actually I was say we

got tons of questions we do have uh certain data structures that I wanted to go over with um so here's uh should we

should we hold up for next week yeah sure you know and um so it's it's actually for me just so exciting that

this first part of this talk happened today um I do a lot of work with uh Marine Science and particularly

understanding uh genomic data about the oceans and just last week um a scientist I work with released the biggest data

set ever of marine genomes with all of the integrated information about their entire environmental situations right so

I I would love to maybe uh touch base over this week and then if if it works for next week we could present some

examples too and and I feel like digging into this would be so much fun further yeah sounds good awesome awesome how a

hand you thank you so much my friend thank you thank you J you know the one thing I always do at the end of my my

talks I do a little selfie yeah oh Jerry that's a beautiful practice

bro self so beautiful sorry Start TI