Building scalable cloud databases
Scaling out databases can be easily accomplished using scalable tools and features for Azure SQL Database. In particular, you can use the Elastic Database client library to create and manage scaled-out databases. This feature lets you easily develop sharded applications using hundreds—or even thousands—of Azure SQL databases. Elastic jobs can then be used to help ease management of these databases.
- The Java version of the library, see Maven Central Repository.
- The .NET version of the library, see NuGet.
- Get started with Elastic Database tools
- Elastic Database features
- Shard map management
- Migrate existing databases to scale out
- Data dependent routing
- Multi-shard queries
- Adding a shard using Elastic Database tools
- Multi-tenant applications with elastic database tools and row-level security
- Upgrade client library apps
- Elastic queries overview
- Elastic database tools glossary
- Elastic Database client library with Entity Framework
- Elastic database client library with Dapper
- Split-merge tool
- Performance counters for shard map manager
- FAQ for Elastic database tools
Scaling out applications using sharding presents challenges for both the developer as well as the administrator. The client library simplifies the management tasks by providing tools that let both developers and administrators manage scaled-out databases. In a typical example, there are many databases, known as "shards," to manage. Customers are co-located in the same database, and there is one database per customer (a single-tenant scheme). The client library includes these features:
Shard Map Management: A special database called the "shard map manager" is created. Shard map management is the ability for an application to manage metadata about its shards. Developers can use this functionality to register databases as shards, describe mappings of individual sharding keys or key ranges to those databases, and maintain this metadata as the number and composition of databases evolves to reflect capacity changes. Without the elastic database client library, you would need to spend a lot of time writing the management code when implementing sharding. For details, see Shard map management.
Data dependent routing: Imagine a request coming into the application. Based on the sharding key value of the request, the application needs to determine the correct database based on the key value. It then opens a connection to the database to process the request. Data dependent routing provides the ability to open connections with a single easy call into the shard map of the application. Data dependent routing was another area of infrastructure code that is now covered by functionality in the elastic database client library. For details, see Data dependent routing.
- Multi-shard queries (MSQ): Multi-shard querying works when a request involves several (or all) shards. A multi-shard query executes the same T-SQL code on all shards or a set of shards. The results from the participating shards are merged into an overall result set using UNION ALL semantics. The functionality as exposed through the client library handles many tasks, including: connection management, thread management, fault handling, and intermediate results processing. MSQ can query up to hundreds of shards. For details, see Multi-shard querying.
In general, customers using elastic database tools can expect to get full T-SQL functionality when submitting shard-local operations as opposed to cross-shard operations that have their own semantics.
Get started with elastic database tools - to try the sample app that demonstrates client functions.
Azure SQL Database elastic query overview - to use elastic queries.
Moving data between scaled-out cloud databases - for instructions on using the split-merge tool.
Not using elastic database tools yet? Check out our Getting Started Guide. For questions, please reach out to us on the SQL Database forum and for feature requests, please add them to the SQL Database feedback forum.
We'd love to hear your thoughts. Choose the type you'd like to provide:
Our feedback system is built on GitHub Issues. Read more on our blog.