IBM CAS Solves Unstructured Data Growth

In this episode, Ray Quattromini speaks with Patrick Kay, Product Manager for IBM Content Aware Storage (IBM CAS) at IBM, in one of the most technically detailed conversations on the podcast to date.

IBM CAS is IBM's semantic interface for AI-ready data, a solution designed to transform unstructured enterprise data into a searchable vector database, without requiring costly data migrations or duplication. Patrick explains how IBM CAS uses Active File Management (AFM) to connect disparate storage environments, how the RAG framework is embedded directly into the storage layer, and why incremental vectorization dramatically reduces the GPU compute required to keep AI applications grounded in up-to-date enterprise data.

They also discuss IBM CAS deployment architectures, storage tiering from flash to tape, the impact of the global storage shortage on AI infrastructure planning, and why IBM CAS is increasingly the missing layer between enterprise data estates and production AI workloads.

Essential listening for CIOs, data engineers, storage architects and anyone evaluating IBM CAS or enterprise AI infrastructure.

Transcript

SPEAKER_02 0:00

Yeah, we got the same shoes.

SPEAKER_01 0:02

Wow. Okay, that's funny.

SPEAKER_02 0:07

Just wear the shoes on so say that's sh that's pretty that's pretty funny, actually.

SPEAKER_01 0:14

That's amazing. My wife got me these shoes, so I guess you have good you got good taste for it.

SPEAKER_00 0:20

Your business is sitting on a gold mine of data, but can you actually use it? We're talking to IBM's product manager for content aware storage about how AI is finally unlocking data that enterprises didn't even know they had. This one's a game changer. Let's dive in.

SPEAKER_02 0:35

Hi, I'm Ray Cotramini from Fortuna Data. Welcome to Smarter Strategic Thinking. Today we've got Patrick Kaye from IBM. Thank you very much, Patrick, for coming in.

SPEAKER_01 0:44

Hey, thank you, Ray.

SPEAKER_02 0:46

Do you want to explain your role at IBM and what you do and how you work?

SPEAKER_01 0:50

I'm the product manager for Content Aware Storage. I've been managing it for about a year and a half now. I sit under the storage software portfolio that's led by Albert Ho. I mean, not so new GM now, but Sam Warner, who took over last summer as GM, leads the storage business into doing a heck of a job. That's for you, Sam, if you're watching. And how long have you been at IBM? Coming up on four years. Started IBM after business school. Actually, in our corporate strategy team, got a sort of a bird's eye view of the company. I mean, all the different business units and various functions. So it was a great place to get introduced for me to the IT industry and to just how big blue works. After a little while, though, I wanted to get my hands a bit dirtier, really dive into the tech and understand it. Storage is about as deep as one can get. An opportunity opened up and haven't looked back. So it's been a lot of fun.

SPEAKER_02 1:39

You look after content-aware storage. Can you explain to our viewers what content aware storage is in IBM's world?

SPEAKER_01 1:46

It's the semantic interface for AI ready data. The way it came about was our CTO uh Vincent Sue, probably about two years ago at this point. I mean, he really saw that everyone is kind of familiar with now, just there's this huge data explosion, particularly of unstructured data. And of course, these AI tools are coming online. And he really saw now we have an opportunity to unleash the holy grail of storage as he puts it, which is actually being able to use natural language to query information and storage. And so that's where the concept of content rare storage or CAS was born. And this idea of us being able to take data, transform it, and index it, and then put it into an AI ready format, and then we can serve that data up to AI applications uh when they need it. Semantic interface for AI ready data. We're just AI ready data.

SPEAKER_02 2:31

What are the system requirements to run CAS?

SPEAKER_01 2:34

Right now, it's the transformation process is a fairly compute-intensive workflow. So at a at a very basic level, we use a couple GPUs in a few servers. We build a little compute environment, if you will, and then we connect that to our storage environment. We use IBM storage scale, which is our parallel file system solution. Been around, you may know it as GPFS. Yeah. What once was GPFS, although a lot of people still call it that. We use that once we do the data transformation, we build a vector database, and then we store that database in the storage scale. It is somewhat compute intensive. So this is a, and I know we're gonna get into this and talk about RAG and all that, but there's no kind of magic bullet, if you will. Like it's a this is a complex data transformation process that has to be has to be done before the data is ready to go.

SPEAKER_02 3:20

And can people run it on their own team? No, absolutely.

SPEAKER_01 3:23

They can deploy it as software. We actually GA'd it um just about a year ago now at GTC 25. It was GA'd as as software only. So for about the first six months, it was only deployed as software, and we deploy it in IBM Fusion. You can think of it as a it's deployed as an AI data service within IBM Fusion. Back in December, so with our Fusion 212 release, we can now deploy CAS natively in Fusion HCI, so on IBM infrastructure. Yeah. And then very, very shortly, uh, we'll have it available on the storage scale system 6000 natively as well. So there's a few different architectures you can select or bring your own hardware, we'll deploy it there too.

SPEAKER_02 4:02

In order for AI tools to work, data needs to be drawn from lots of different locations. We've got cloud, we've got on-prem, we've got pools of storage, we've got we've got it spread everywhere. How does CAS help in this situation? Traditionally, how does it happen and what does CAS do in terms of pooling that data?

SPEAKER_01 4:19

We leverage an existing piece of IBM IP that was that's really born out of IBM storage scale that I referred to earlier, called Active File Management. It's a really cool piece of technology. If you have been following storage scale through the last few years, there's a bit of a messaging rebrand as the global data platform. Yeah, we want to create this single pane of glass where you can connect all your different storage endpoints to effectively, and then pull that into the storage scale environment, but one-stop shop for consuming storage from there. And that was done using AFM. It allows you to connect third-party storage systems, S3, NFS capable, continuing to build out a couple new connectors actually. But what that does is it allows us to say tap into a Net App or a Dell solution, yeah, or or cloud via S3, pull that into the data platform, and then make it available for applications. In this case, the application happens to be CAS. Yes. So we suck in that data, we unify these data silos, if you will, and then we can process that information with CAS and build out our knowledge repository, the vector database. The very special thing that we do, we don't require a full copy of the data or data migration. We're actually caching this data in, processing it, and then we're able to get rid of that cache. So we're just processing essentially like a representation of that data and keeping that. And this, of course, will result in massive savings and reduced headaches. And you're not doing this data migration. You're you're just connecting the data sources, and from there, AFM takes care of the rest.

SPEAKER_02 5:48

If you've got a petabyte in the cloud, you don't have to pull that petabyte back to look at it. Correct. Which is a big saving from an eagerist point of view, isn't it?

SPEAKER_01 5:55

Absolutely. Other advantage with what we've done with AFM is they have a watch folder capability. So we're essentially monitoring for the changes in that source repository, say in an S3 bucket. Yeah. And when that data changes, we can process just the deltas. As we often found, there's a lot of batch processing in the RAG workflows, but this allows us to say, we'll take the petabyte example again. Let's say only, you know, over the course of a month, only maybe 10 terabytes of that information changed. We're only going to process that 10 terabytes, and we're able to have that flow through the solution. Yes. This results in massive efficiency gains.

SPEAKER_02 6:28

And because we're not pulling back a petabyte from the cloud, we're also saving a lot of time.

SPEAKER_01 6:33

Yes.

SPEAKER_02 6:34

Can you explain to our viewers what RAG is and how RAG works?

SPEAKER_01 6:37

Rag is retrieval augmented generation. At a high level, it is a framework for getting enterprise data to AI applications, to LLMs, really in particular, and giving those LLMs that information at inference time, you know, when you're actually prompting the LLM to produce new information. So what that means practically is in this retrieval part of it, I've got some data somewhere. It's in some, it's in some format, maybe it's a bunch of PDFs or or PowerPoints. First, I need to get that data AI ready. And that involves a this transformation process. So typically that's I gotta take this PDF, I need to, you know, parse this information, and then I need to create a chunk. So that might be say, for one page, you create one chunk of information, and then I create what's called a vector embedding of that chunk. So this is a multidimensional mathematical representation of that data, and it really captures the meaning of that data. So we'll be talking about semantic search during this. What that means is you can perform a search. It's almost like a really, really fuzzy search. I can search based off of the underlying and concepts and information in that particular document. So it allows for natural language search, effectively, which is funny enough how AI likes to speak. So now we create this vector vector representation, we put it into what's called a vector database, and now that data is ready to be retrieved. Now, when I'm using my LLM, just think of think of I'm using a say a chat GPT or CLOD or something. I type in my question. The application is then going to do is they're gonna take my prompt, they're gonna parse it and send it to the RAG application, and then they're gonna perform a search against the vector database, and they're gonna pull in information in what we would call vector space. So in kind of the similar part of a vector space, pull that in, and then that gets retrieved back to the LLM and put into the context window. So now we've augmented the LLM response. So that's the augmented part. And then as it's generating its new information, it now has this additional context to work with. It allows the LLM to produce an answer to your query or your prompt that is now grounded in the truth of your enterprise data. Right. So it's a bit of a ramble, but that's effectively what RAG is doing, and it's it's really important because that is the magic sauce of CAS, is embedding this RAG framework deep into the storage layer.

SPEAKER_02 8:52

Typically, when you run a chat GPT search, you type in the keys, so what is the weather in BasingStok like today, for example? Sure. The way that your software looks at it, it doesn't actually, every time it doesn't load that whole command line, doesn't it? Because it already knows what weather is, it knows what where BayesingStok is, it understands that context. So you're not loading the command strings and everything to start from scratch. Is that right?

SPEAKER_01 9:17

But that analogy, that that's really probably just doing a re a web crawl at that point. Like that's OpenAI has some web crawler when you type your prompt in and it's a tool call, it's gonna go search the web. I think what you're describing is kind of this KV cache concept. So you've already processed an answer. We can store the results of that answer in storage and then recall it at inference time to save on computation costs. Yes. The RAG framework would be I've just have this knowledge repository. And so the application at that point doesn't necessarily know what's in the knowledge base, it's just tapping into it. So if you did happen to say, I want the weather from you know, embasant stoke from a year ago, and that was information that you stored in the vector database, then it can go pull that into the response and give you an accurate answer instead of hallucinating and just making something up.

SPEAKER_02 10:00

Yeah. How does the incremental vectorization architecture fundamentally reduce computational costs and latency compared to traditional rag batch processing?

SPEAKER_01 10:10

We talked a little bit about AFM and its watch folder capabilities. We hook up all these different data sources. Sometimes I like to think of it as we're stretching across the entire data estate, unifying all these data silos. I think of it as a phase one and phase two process. Phase one is initial ingest. So whatever the petabyte example from earlier is, we're just processing all that petabyte of information and indexing it into the vector database. Phase two is the incremental ingest. Yes. You could almost think of it as real-time or near real-time data streaming. We're capturing those changes. You can imagine if, say, it's only changing 1%, 1% a month. Now we're um now we're only processing what would that be, 10 terabytes, right? Of information. So instead of processing in a batch mode a petabyte of information, say every month, let's say that's your SLAs. Like I need my database to be refreshed at least monthly, you would have to process a whole petabyte of information versus just the incremental of the 10 terabytes. Because it already learned exactly it already knows the vast majority of the information, and that didn't change. The computational requirements to process a petabyte are a lot more than 10 terabytes. Yes. Just from an infrastructure, it's a fairly linear relationship. If you needed 20 GPUs to be able to process that petabyte information in the time frame that was acceptable to your, you know, to that organization, then you can imagine, well, I probably only need maybe because CAS is a minimum of two GPUs. Now I just need two GPUs to do the incremental ingest. So in that little example, you just save 90% of your your your compute requirements. Yes. So that's a huge savings, obviously. It's just a much more efficient way of running your processes. And you're probably familiar with IBM bought Confluent recently. And so this idea of kind of data streaming and and controlling the data flows through a through a platform. I mean, IBM is sort of making a bet on that. It's just so much more efficient to be able to capture these changes in in real time to do incremental versus batch. When we were designing CAS, we we learned from a lot of clients and even internally at IBM, that's how RAG was working. It was batch processing.

SPEAKER_02 12:08

We've got a petabyte, which in the grand scheme isn't that much data today. It fits in about this much space. Let's say this this organization has processed the hundred percent of the data using their 20 GPUs and everything else. Now we're going to use CAS to do the incremental changes. So we need two GPUs. Could we then on the next petabyte take those 20 GPUs and start processing the next batter data and then use CAS to then start? Is that you certainly you certainly could. So a bit like farming, you know, we need to fry our seeds, right? Okay, that field's done. Let's go to the next one. And then sure.

SPEAKER_01 12:42

I I mean I think you're I think that's a great point. And something we can we can touch on too is like we see is that initial that initial ingest pool that's gonna be so much uh more in magnitude in terms of how much you need to stream through versus the incremental part, yes that you're gonna need additional resources up front. But once you're into the phase two incremental processing stage, you're gonna want to offload to another task. So maybe it's another maybe it's another CAS pipeline that you're setting up. Yes. So that could be one thing, or you can send those GPUs back to your inferencing architecture and then do more token production. And the one thing we talked about during the London Storage Days event in March, the value metric, and you'll hear Jensen talk about this when it comes to AI, is the token.

SPEAKER_03 13:24

Yes.

SPEAKER_01 13:25

So every time that a GPU is not producing a token, you're you're losing value, but you want to produce valuable tokens, and that's what CAS helps you helps you do, is to produce high value tokens grounded in your data. But your I think your intuition is spot on. You should absolutely have a plan for how to maximize the usage of your GPUs and get the most value out of them as possible.

SPEAKER_02 13:48

It's like a hub and a spoke analogy. CAS is your hub, and then you send out the spokes, which could be your 20 GPUs, and then they do the search on the various pools of data that you're trying to analyze and pull it in one place. And then what happens is you start using one of those spokes, could be two GPUs, and then that then they just move them around, you know, like a hub and a spoke type effect. It's hard to visualize and see, but I I get the concept. You can make far better use of your resources and the cost that you've spent on those resources, and you can plow through that data much faster.

SPEAKER_01 14:19

Yep, absolutely. As opposed to where we started, if it was batch processing, you always need those 20 G batteries dedicated to the batch. To the front end. Yeah. Yeah. So it's a very easy way to think about how you're saving and saving an infrastructure requirement.

SPEAKER_02 14:34

How does the architecture handle multimodal data, charts, tables, images during the ingestation process?

SPEAKER_01 14:41

That's really all about the AI data processing pipeline, or you'll probably hear it referred to as the RAG pipeline. And we have a couple different options you can select from an IBM open source project called uh called Dockling. As I understand it, is IBM's fastest growing open source project. That's a document processor. We use that as the ingestion for one of our pipelines. Uh, the first pipeline we actually released, though, was in collaboration with NVIDIA. So we took their MV Ingest pipeline, which accommodates multimodal data. NVIDIA uses this concept of NIMS, NVIDIA inferencing microservices. So they take different tools, containerize them, and then you can build these data processing pipelines. So the NVINGEST pipeline will be one of them, and there's a series of another, really six to ten other NIMs that you can deploy in the pipeline that perform some of the data preparation tasks. So it might be like the OCR NIM, for instance, extra chart extraction NIM, a table extraction NIM. Um, and we can deploy these in sequence and then process the data in a multimodal fashion. And so it's really all about how that pipeline is designed. So it can handle spreadsheets? It could, but uh, you know, I would think of a spreadsheet typically more as a piece of structured data, and we're much more tuned for unstructured data. Right. I think if you're doing really if you're doing structured data processing, I would I would typically refer you over to our What's Next.data teammates. In theory, sure. You could definitely it's it uh you could you could certainly do it. You could you could definitely get a spreadsheet in there and and and and vectorize the information.

SPEAKER_02 16:13

If you've got if you want to know what who your highest performing customer is and your lowest performing customers are, and you've got a hundred customers, you could say to the to the database at the end, you know, who are my best customers and my worst customers, and what's their average value? And that's a good point. Do I want to get rid of the bottom feeders because they create most of my take up most of my time and they're a lot of hassle? Whereas the top guys that don't need any help.

SPEAKER_01 16:35

So for a hundred, you could absolutely do it. Uh that's that's that's no problem. We're talking maybe billions of rows and columns? This gets a little more, a little more dicey, probably, but yeah, I I think for kind of you know daily spreadsheet tasks like that, especially in a small uh maybe a more smaller, mid-sized organization, that would easily get processed.

SPEAKER_02 16:55

Can it can it hook up, and I don't know the answer, can it hook up to databases?

SPEAKER_01 17:00

We don't really hook up to databases, not not with CAS. It's really more of these storage repositories, S3 and NFS capable. There is a plan, it's on the roadmap, as we product managers like to say, uh, to be able to tap more into content management systems. Yes. So you can think of box and OneDrive or FileNet is one we keep getting asked about from the IBM perspective, of course, as a full IBM solution. So beyond just what I'm working on with CAS, we could definitely architect that kind of solution for a customer that includes some kind of unstructured data processing engine like CAS with more structured data management capabilities with with the with the Watson X portfolio. So that's certainly doable. It's just not necessarily in in my product.

SPEAKER_02 17:42

What is the realistic scalability limit of the vector database and what are the performance trade-offs, latency versus recall? It's like 100 billion vectors.

SPEAKER_01 17:51

I I'll first I'll uh I would encourage you to go read a recent blog. It was published last week, actually, by our our research team testing vectors vector database scalability. We got up to they got up to 100 billion with subsecond latency and still having over 90% recall or accuracy. So that was that was a pretty pretty promising test. Now that wasn't a test environment and the real world typically typically looks different. Often what I see nowadays are I mean, customers, even though we GA'd CAS a year ago and RAG is it's I wouldn't say it's immature. I mean it's it's still maturing. We talked to talk to customers, we're still playing around with relatively small amounts of data that would only turn into maybe tens of millions or hundreds of millions of vectors. That's easily containable. We can certainly get up into the to the single billions and and and beyond. An element of stress testing the um uh the system, if you will, that's gonna kind of advance in parallel with with customer uh the customer willingness to put more and more of their data through rag frameworks like uh like CAS. I mean, if you're only talking about a few terabytes of information, easily containable, that's that's no problem. When you get up into the petabytes of information, that's when you really need kind of an engineered solution like like CAS to be able to deal with. Overall, I'd say your intuition is right. I mean, the the more vectors that are created and you have to index, there's gonna be trade-offs in performance. Yes. From the database perspective, is you know, how quickly am I getting am I getting my query? How fast is my retrieval, the latency aspect, and how accurate is it? That's another thing that folks probably will have to start learning a little bit more when it comes to semantic search versus keyword search, is it's not exact. And so you need a system that can, I mean, usually maintain a r uh above 90% accuracy, I'd say is is pretty safe, but that still means you're it's not it's it's not a hundred percent. But there is that is a kind of a a trade-off of doing semantic search versus just pure keyword search like we're used to with databases. Yes.

SPEAKER_02 19:56

Obviously, CAS connects to your storage. What security rules and things are in place to protect that data from an RBM podcast perspective?

SPEAKER_01 20:08

We ran into this really as we were just designing the solution. And we heard this not only from uh you know a number of of customers we were speaking with, uh also internally in our CIO office, but also with NVIDIA. And NVIDIA, if you if you talk to them, they keep hardy harping on this, they call it the G-word, governance. Data governance is super uh is even it's hard to say it wasn't important before, but it's even more important in the world of AI. Because now you you you know you might not even eventually we're it's gonna be automated and you won't even necessarily have a human watching these watching these these data flows. Knowing knowing that up front, we designed um this this concept of being able to preserve the access control list, preserving existing security policies that you had from your existing data sources, say in your in that S3 bucket, yeah. And then being able to flow those through in the dirt through the vectorization process and up to the actual higher level application. So we built that in natively into CAS. The other thing is and this is a great reason why you should build a solution like this in the storage layer, is we get to take advantage of the file level security policies that are already there in IBM storage scale. So once that information is in the in the file system uh and we build those security policies, we basically just they basically just uh come along for the ride, if you will, with CAS. So those are two things we've we've done um to improve the um the security. Now the governance piece, and this is really important not only from a data control perspective, but also an efficiency a sufficiency standpoint. A few years ago, IBM uh released a a product which we we now call the Fusion Data Catalog. And now that also falls into my my purview, but I'll we'll just call it the CAS umbrella. So what is what is Fusion Data Catalog? That's a metadata Tagging and filtering solution. And what we did, and I I wish I could take credit for it, I'll take some as the product manager, but the engineers were the ones that really came up with the idea and said, hey, why don't we put this on the front end of CAS? And what would that allow us to do? Oh, that allows us to, you know, index the metadata and then be a little more discriminatory about the data that actually flows through the vectorization process. So let's say you had some sensitive information, PII tags or whatnot. You don't want to vectorize that. You don't. So now you've added a layer of governance into the CAST process. It also turns out the added benefit is that it makes your solution much more efficient. Because let's say I'll even take a conservative estimate, we filter out 50% of the information that's coming through for what for whatever we go. Could be controls, could be whatever rules you've policies you've set, could be duplicates, all these sort of things. Now you just saved 50% of your processing requirements as well. So you just made the solution more efficient while making it safer for the customer. So that's really the third thing we've done. So we have our access controls, you know, RBACs and whatnot, RBAC and whatnot, file level security, and then metadata filtering. And leveraging these, we're able to make a pretty finely controlled system and being um very um uh you know being very discreet in how we get the data to the right user, uh, and that only the right user is gonna have access to to certain data.

SPEAKER_02 23:29

That's pretty impressive. I didn't even think about that. But it's as you say, it's the fusion data catalog at the top. So do you use do you create a table of do not import this at the beginning and then it and as you process it's right the other way.

SPEAKER_01 23:43

Exactly. You could you can build exactly that. Yep. I mean it's user, it'll be user policy-based, so you can design it how you wish, but that's that's effectively what you're doing.

SPEAKER_02 23:52

What is the specific mechanism for connecting CAS to existing third-party storage? Is it you just need to map the storage to the CAS or how does it work?

SPEAKER_01 24:02

Yeah, that that really goes back to the to the AFM, the active file management technology we were we we we spoke about, we spoke about earlier. So AFM is really that is really the is really the magic in this case. It's what allows us to uh connect to these third-party storage systems, and it has a built-in watch folder mechanism that is monitoring uh for those incremental changes and then processing those in. So you can think of really CAS, if we have our kind of our file, our file system layer, which is scale, you know, that's actually what's all the data is being sucked into, and then once it's in um the file system, really in the cache of the file system, we're able to then process it through content aware storage, um, build the vector database, and then we're able to you know dump things the cache and then and just keep repeating that process until we've indexed all of the data. While of course, not copying or migrating uh the data in in in actuality. So all the data just stays in place, it just stays where it is. There's no need for an expensive job or a bunch of man hours or whatnot. Or extra storage that you can't buy. Or extra storage, which IBM would would love you to pay for, but but we'd prefer to make a a more elegant solution for the customer than than have you pay double what you need. Another reason why when you're doing these vector transformation projects or rag projects, why you need to take storage into account. The reason is through the vectorization process, you end up creating new data, these vectors. Yeah, and in our case, you might you you also have these chunks that you're creating as well, and you you you may want to want to store those for the future, these kind of data derivatives as we like to call them. And you know, our right now for text-only data, um, it's about a it's about a 10 to 3 ratio, which means for every 10 terabytes of source data, we end up effectively creating three terabytes of new data in the in the vector database that now has to sit in storage.

SPEAKER_02 26:05

Right.

SPEAKER_01 26:05

So you can imagine, and it surprises some customers when we're sizing these solutions, like, hey, we we have a petabyte of information, and so we're just gonna, you know, we just process it and we're good to go, right? Like, no, actually, you need to buy another 300 terabytes of storage from us because this is this is just the reality of this process. And that was text-only data. If you if you talk to NVIDIA, I mean their their initial testing they were saying up to 10x. Wow. Which is kind of astounding. Yeah. And in an early testing, we were sitting, we we found somewhere around the two to three X mark. Um, and that but that tends to be a little bit more for multimodal data. So text only is it that's a 10 to 3 ratio, but you can imagine it might be more like a a one to a two or even even higher ratio of of data creation. And so you really have if you're trying to do this at scale, you really have to do it at the storage layer. Um, otherwise you're gonna run into some some problems very quickly.

SPEAKER_02 27:02

Could we tier it? So obviously, we all know there's a global shortage of storage.

SPEAKER_04 27:07

Yeah.

SPEAKER_02 27:08

Of RAM, of of pretty much everything at the moment, and it's not there now, there's no lights at the end of the horizon at the moment. So could we could we push some of that data to IBM tape, for example?

SPEAKER_01 27:19

I feel like you're setting me up here. Yeah, well that's my job. It's a great question, and the answer is absolutely. So storage scale, we've been talking to and and really what CAS is leveraging as the as the storage environment. It has the storage tiering capability built into it, so we can uh we can uh you know tier from from flash to disk to tape. That's that's built into the technology. So we can do the same thing, the same thing with CAS then because we're because it all sits in scale at the at the end of the day. And so absolutely we can use that storage tiering capability to put the really cold stuff off into tape. We can we can even tap into tape data relatively soon. Again, uh on the roadmap item. We have uh with with uh AFM we have S3 glacier and S3 Glacier capability, which allows us to connect to you know to to the to information in tape. And so we'll be able to tap into that you know uh later later this year and pull that into CAS. So now we can re really wherever your data happens to be located, we can we can pull that into the into the environment. Um so yeah, your question is you're absolutely spot on. I mean, if you are if I mean data explosion, yeah. We have this uh demand shock going on with memory and and and and and flash, even disk. And who's who makes the the best uh enterprise you know tape tape system in the in the world right now? Well, that's that's IBM. Just so it's almost what a man, what a great strategic move that was to make sure that we had had tape um available.

SPEAKER_02 28:57

So so what are the is it the diamondback 61 petaberytes? Yeah, I forget the exact amount, but a lot. And then that's got the S3 connector, so it just sits on the S3 Glacier, yep.

SPEAKER_01 29:09

Yep. It's a lot. It's a lot.

SPEAKER_02 29:12

It's a lot of data. But in terms of we could we could buy that within a month rather than waiting six months, twelve months for storage, you know? It'll be a lot cheaper. Yeah, it would be a lot cheaper, isn't it?

SPEAKER_01 29:22

A lot cheaper per terabyte pricing. So something to think about.

SPEAKER_02 29:27

How does the appliance model reduce the need for specialized AI engineering skills compared to DIY rag? You know, you said build it yourself or bring your own. So what what are the trade-offs here? Other than trying to get the kit in the first place?

SPEAKER_01 29:41

Other than getting the kit in the first place, if you're if you're lucky enough to get it. You can deploy this as software only. So we've done, you know, we've we it on on your own hardware, that's fine. And so it's still, you know, it's still not real, it's still not DIY in that case. You're installing it on your hardware and you get the you get the value out of it. Why might you want it on the IBM appliance model, whether Fusion HCI or the storage scale system 6000? The high-level reason is really just time to value. You click the button, it gets deployed into your data center. IBM has an expert lab services that installs everything and makes sure it's ready to go. So you are at hands-on keyboard, hello world experience, you know, much faster. Um and it's really this kind of white glove, this white glove experience that IBM's delivering for the customer. So that's that's really the the the high the high level reason. The second is while it is all x86 architecture and NVIDIA GPUs, which are really, you know, really the most standard in the industry for um you know accelerated computing at the moment. On the fusion HCI architecture, you know, it's we've we validate it and we test it and we stress it, and so we can really stand by that on that's on that specific hardware that we're delivering, it's gonna work. And I mean, you know how this this happens.

SPEAKER_02 31:01

It's it's software and it's supposed to just run anywhere, but incompatibilities and driver issues, and you know, we've all been there.

SPEAKER_01 31:08

It's yeah, it's probably just gonna run a little bit more smoothly if you buy the IBM hardware, get it all in a box with a bow wrapped around it. It's just gonna, it's it's just gonna be a better customer experience. It saves on head count, the head count costs, and the CIO pulling their hair out um because of cost overruns, because of data migrations, and and then the data scientist doesn't really like why would this is getting into why wouldn't you want to do it DIY? Well, the data scientist doesn't want to build and maintain rag pipelines. I mean, they want to do the the next level, yeah, the next level kind of you know analytics and whatnot. And you got platform engineers having to figure out where they're gonna deploy all these, you know, deploy all these different functions. Um you have an AI application developer who I mean they just want to tap into they just want to tap into the vector database. They just want the data.

SPEAKER_02 31:58

They just want to get on with that job.

SPEAKER_01 31:59

Exactly, which we make very easy because we built an API and an MCP server so you can tap into the information that way as opposed to having to write a script directly into the database um for your AI application. So that that that AI app developer, I mean, they just want it to work. And when it doesn't work, they're gonna go yell at the data scientist or the data engineer, and that data engineer is probably yelling at the platform engineer. The storage administrator, frankly, is usually not really involved in this process at all. They're like, what is Rack?

SPEAKER_02 32:25

Yeah, yeah.

SPEAKER_01 32:26

And so we have we've developed a way to make this the storage admin the hero of the story, if you will. Like, hey, all this stuff, didn't you know IBM has an engineered solution that just does all of it out of the box and solves all of these problems? Oh, and it does it more efficiently, and you get you know your 24-7 support, and they're gonna come and install them for us, and and they have the hardware to do it. I mean, you get so many benefits. I mean, you can always you can always slap a number on it, but at the end of the day, I think it comes back to that time to value. And especially in the tech world today, I mean, that is of utmost importance. Like, how quickly can I get up and running and get value out of these AI tools and solutions? I mean, the competition's doing it, so you better you better figure it out.

SPEAKER_02 33:09

And and the other thing is you you eliminate the incompatibility issues. Whatever you buy, whether it's this brand or that brand or whatever it happens to be, it all went there could be a BIOS issue, there could be a driver issue, you know, and these these things, and they all add to delays. It might ultimately get it working, but it might take you six months as against one month, you know. So that's the trade-off, isn't it? And and if it goes wrong, or something overheats because it's not cooled enough, you know, then you start pointing fingers, don't you? And this is how it goes.

SPEAKER_01 33:35

I can see in your face, Ray, you've dealt with this a time or two.

SPEAKER_02 33:39

Just once or twice in my life. What types of queries does the API support beyond semantic similarity? E.g., hybrid search, keyword search, parcel string matching. How does that come into it?

SPEAKER_01 33:52

So we designed uh it's it's essentially a hybrid search mechanism that you that you touched on there. So we're it's a combination of semantic and and keyword, which results in ultimately just a more accurate response to uh gets sent over via the API. So you're doing that that semantic, so that vector that vector matching, you're also taking that prompt and parsing out the keywords and running that against the database. And then we have our uh a re-ranker model that takes you know the top K responses, says, here's you know, let's say it was the best 10, and then I'm gonna shoot those over to the application. It ends up being being pretty good. Like we talked about that that that recent test that our research team did, over 90%. It's probably a little bit higher, but they're only comfortable saying 90. That hybrid search ends up being pretty darn good for our purposes.

SPEAKER_02 34:38

Where can a viewer of this channel find out more and have a discussion on demo of Cass? So if anybody's watching this video, what's the best way for them to understand more about Cass?

SPEAKER_01 34:48

Go to IBM.com into IBM Storage. You can find there in either through Fusion or Scale. There'll be links to to content aware storage. You can read more about it there. Shout out to me, please. You can yeah, reach out to reach out to Ray and the team or your friendly IBM seller. I mean, we've got a bunch of them in the in the UK and in AMIA. Um, so please reach out and we we'd be happy to to work with with any customer and and show you demos, run POCs, and really get you comfortable with this new technology. And the stuff is it's moving at at lightning speed, and and we're trying to make sure that it's done in uh you know a safe, methodical way, so it's actually usable for the enterprise.

SPEAKER_03 35:27

Yeah.

SPEAKER_01 35:28

It's a must-have. I mean, the the the data protection is enterprises are are not going to equivocate when when it comes to that. So you have to build that build that in and make make sure that the data being presented to the AI um is uh is is that's safe to do so. Because otherwise the AI will take it and run with it, and you you don't want to be the one with a big a big data breach.

SPEAKER_02 35:51

I'm a business, I want to invest AI. I you know, yeah, I've got lots of data on lots of pools, cloud, on-premise, wherever. I can't use AI because my data's not in one place. Sure. Can I use CAS and AFM to pull that data, make some sense of that data, and then create an output?

SPEAKER_01 36:14

Absolutely. I mean, that's ex I would I would say that's that's precisely what we designed it to do. Connect all of these disparate data sources, these data silos, into one pane of glass or or kind of a global data platform, as we like to call it, and then CAS will process all of that, build this this knowledge repository out of the vector database, and now you can tap into that database through our API with whatever your application architecture happens to look like. So that's that's a great question. That's precisely what we we designed CAS to solve, and it's a really difficult problem to do, yeah, especially at scale.

SPEAKER_02 36:51

For me, I've been in storage decades, and one of the things that I see is historically a custom would buy some storage to solve the accountants need to run a new accountancy database, or the database administrator wants a new database admin system, you know, and they buy these pools of storage that sit all over the place, but no one knows what they really do. They spin for 10 years, and like, oh, can we power that off yet? You know, and and no one can make out a title of what they've actually got, and it grows and grows, doesn't it? Yes, yes, and this is part of the problem that I'm asked frequently is we we don't have the data in one place to make any sense of it.

SPEAKER_01 37:29

When we were designing Cash, going back to our CTO Vincent, I mean, this was a problem he he's been in storage for for ages as well, probably longer than I've been alive. Um he knows his stuff. And um, and and he had been telling me a story as we were designing CAS that you know there is an insurance, an insurance company in the US, anything under a million dollars, they would just settle because it wasn't worth their time to do the data discovery.

SPEAKER_02 37:54

Wow.

SPEAKER_01 37:55

And so Cass can solve that problem. I mean, now it's just simple natural language search to help you understand of, hey, is this a reasonable claim that we should we should we should pay out, or is it or should we, you know, should we dig into this a little bit more? So that was one that was one really kind of uh harrowing example of like it's worth a lot if you can figure it out.

SPEAKER_04 38:15

Yeah.

SPEAKER_01 38:15

I mean, and you think of the volume of claims that they're probably settling just because it's not worth their time now that it and now it could be. On the other side, from a more just conceptual standpoint, one of my colleagues, Pete, he's he's really been been pushing this concept of unlocking dark data. And dark data is our our definition of it is kind of exactly what you're referring to. You have all these large pools of information sitting in a storage somewhere you don't even know what it is. Um let's bring it all together, let's through either through either our you know our fusion hci or storage scale system, let's pull it in with AFM and with CAS, we can now make sense of it. And that's that's gonna unlock a ton of value um for the for the enterprise or really any organization that uh that's dealing with that data sprawl and just all these all these environments that have been been built up. This this storage debt, if you will. Yes, that's built up over 30, 40 years in some cases at these at these big big companies. So yeah. That's been a real pleasure. Thank you very much for coming in. It's been a pleasure, it's a great excuse to come back to the UK and get a few more full Englishes in me. And uh I've had a great time. Thank you very much again.

SPEAKER_02 39:29

Yeah, thank you everybody for watching. Thank you. If you think this podcast is worth liking, please give us a thumbs up. If not, you've got any questions, speak to me or Patrick. You've got my details. Thank you again.