Updated: AWS has previewed a new bucket type for S3 (simple storage service), designed for AI workloads, and positioned as a lower-cost option for vector data where performance is not critical.

Vector datasets are ideal for providing domain-specific data to LLMs (large language models), with support for similarity search and nearest neighbour queries. The AI technology wave we are experiencing has resulted in vector capabilities being added to popular relational database managers including PostgreSQL, MySQL, SQL Server, and Oracle. 

This now includes S3, even though this AWS service was not originally designed as a database manager. That distinction began to blur with the introduction of table buckets late last year, intended for data lakes. There are now four S3 bucket types:

• General purpose buckets: the original bucket type for any kind of object
• Directory buckets: for hierarchical object storage
• Table buckets: for tabular data in Apache Iceberg format
• Vector buckets: for vector embeddings used by machine learning models

Use cases for the new service include similarity search for medical images to assist diagnosis; copyright infringement search; de-duplication of images; searching video content; semantic search of enterprise documents; and recommendation engines.

AWS states that S3 will automatically optimize vector data for “the best possible price performance for vector storage” and the new service appears to be focused on cost effectiveness, with the claim that the cost of uploading, storing and querying vectors could be reduced by “up to 90 percent,” though without specifying from what.

We asked Andy Warfield, AWS VP and distinguished engineer, about the 90 percent claim. He told us that when doing similarity searches in high-dimensional spaces, “a lot of the data structures are really memory-intensive … to make the index look-up go fast you typically need the whole index to be in RAM or SSD. When you take that type of data structure and put it on S3, where you’ve got hard drives involved, your vector lookups might take minutes or hours to satisfy. So we looked at a different class of algorithms for S3 Vectors that is better suited to S3. It does parallel reading of data from S3 and then filters the content out, so it plays to the strength of S3. Because we used this different class of algorithms and built S3 Vectors storage around hard drives, we’re able to hit a much lower cost basis.”

The trade-off is that performance is less good than with in-memory or SSD storage, but still good enough for many cases. “It’s low hundreds of milliseconds per query, and a query rate in the namespace of low hundreds of queries per second,” Warfield told us. The consequence is that “you probably don’t want to use that in an application that needs to do a bunch of round trips to a vector store to go and answer a question,” he said. “But if you’re doing a search, or RAG (retrieval-augmented generation) as part of a chat bot interaction, it’s a fraction of a second.”

S3 vector storage customers pay separately for adding data ($0.20 per GB), storing data ($0.06 per GB//month), and querying data (a more complex calculation based on the number of requests and the amount of data processed). In the AWS pricing example the query cost far outweighs the other charges. With 400 million vectors stored, 40 vector indexes, and 10 million queries, a hypothetical customer pays $996.62 for the queries, $78.46 for data input, and $141.22 for monthly storage.

The new service integrates with other AWS products, including Amazon Bedrock and Amazon OpenSearch (a fork of Elasticsearch). When integrated with OpenSearch, which also supports vector data. AWS suggests a tiered strategy, with vector data in OpenSearch instance storage for best performance and in S3 Vectors for “near real-time access.” OpenSearch pricing is also complex so the cost comparison with S3 Vectors is not trivial to make. Another issue is that the tiering is not automated. Warfield said: “There’s API support to take a namespace and hydrate it into OpenSearch, so it instantiates the OpenSearch instance, moves the vectors into it, then you can query OpenSearch.” At this point the performance will be equal to any OpenSearch query, but the cost advantage is lost.

“There is a bit of hands-on work because we don’t have deep automation between these two layers,” Warfield acknowledged. “There’s a bit of friction for developers that we’ll want to make better.”

S3 Vectors also support metadata in the form of key-value pairs attached to each vector. A vector bucket can have up to 10,000 vector indexes and each index can have up to 50 million vectors.

AWS has done this work not only to support AI workloads, Warfield told us, but because vector databases are increasingly interesting for a variety of use cases. “You could use this to do data quality analysis, or fraud detection, one of the early applications was cancer diagnosis … we’re just seeing the beginning of what’s possible with vectors.”

Updated to add at 0704 UTC on July 17:
Warfield got back to us with more detail on the claim of up to 90 percent cost saving.

He told us, “Traditional vector DBs are anchored in RAM and SSD, and they have you running a compute instance for your database, even when it’s only being used lightly. S3 vectors anchors cost in storage and assumes that query demand fluctuates over time, meaning you don’t need to reserve maximum resources 100% of the time.

“So when we look at cost, we assume you pay for storage most of the time, while paying for query/insertion costs only when you interact with your data. When we looked across customer workloads, we found that the vast majority of vector indexes did not need provisioned compute, RAM or SSD 100 percent of the time. For example, running a conventional vector database with a ten million vector data set can cost over $300 per month on a dedicated r7g.2xlarge instance even before vector database management costs, regardless of how many queries it’s serving. Hosting that same data set in S3 will cost just over $30 per month with 250 thousand queries and overwriting 50 percent of vectors monthly.

“For customers who have workloads that heat up, they can also move their vector index up to a traditional vector store like Open Search, paying instance-style cost for the time that the database is running hot.”