Handle Cross-Region Databases Connection with DBResolver Library
Connect to multi databases and cross-region databases in the Golang application
A month ago, I was working on a Golang project. This project is related to some authorization data. To make it short, this project must be deployed in multi-region because of the needs of our services.
We use a global database with replicated database for all regions. In our case, for example, our application is deployed in us-west-2 and ap-southeast-1 region. If drawn into a diagram, it will look like this:
So there is nothing wrong with this. But the problem is, when handling the connection separation, I take time to understand how to do this in Golang with ease. I searched for a library that can help with this, and I found one, DBResolver from Gorm, but the problem is that we’re not using Gorm. With some tradeoffs, I decided to write my library that will utilize the connection separations for the Database in Golang.
Introducing Golang dbresolver for Built-In Golang sql.DB
Moving from the abovementioned problem, I created my own Golang library to handle database connections.
Use case 1: separated RW and RO database connection
Imagine you’re building an application (e.g., a media platform like Medium.com) with heavy read operations (query) to the database. Let’s say your database limit connection is only 100, but you have a lot of concurrent users coming in and eventually using all the maximum connections to the database. This will impact performance and user experience.
How to handle this easily is by replicating the DB and separating between the “Write” and “Read” databases. In this case, we will define one Primary (master/leader) database and create other replicas depending on needs (maybe 2–3 replicas for read-only).
So to summarize:
- You have your application deployed
- Your application is heavy on reading operations
- Your DBs replicated to multiple replicas for faster queries
- You separate the connections for optimized query
Use case 2: cross-region database
This is a similar use case to use case#1. The difference is that in use case #1, your application is only deployed and running in 1 region. In this use case, you need to serve a multi-region application but still want to make it consistent across regions.
Let’s say an authorization and authentication service may be an example for this use case. Since it is heavy on reading operations, e.g., Select User info, get user permission list, get user role, etc., and since it’s an authorization system, it should be available globally. So if we block some users’ access, it should be blocked in every region. That’s why we need to ensure the application is deployed across all regions.
- Your application deployed to multi regions.
- You have your Databases configured globally.
Using dbresolver
Based on the use cases I mentioned above, building it with the builtin *sql.DB will take time. Worry not. I created this library to save you time when facing similar use cases.
Here’s the main features:
- Clear differentiation between Primary and Replicas.
With this library, you can define your primary and replicas explicitly without worrying about wrong configurations. - ReadWrite to Primary, ReadOnly to Replicas
All write operations will automatically go to the Primary and read operations to replicas (currently only support round-robin if has using more than one replicas).
So whenever you call these functions (Exec,ExecContext,Begin,BeginTx) in your application, it will automatically be using the primary database. But if you use these functions (Query,QueryContext,QueryRow,QueryRowContext), it will automatically be using the replicas database.
Example With Use Case 1: Multi Readers in One Region
To give an example of how to use it with a multi-reader database, here’s the code:
It’s pretty clear here — we only need to define all of our readers' connections. And initialize the library. After that, we can use it for our database operation in our application.
Example With Use Case 2: Multi-Region Application
And this is an example of how to use it with multi-region applications.
This example is quite different. With use case #1, we need to declare all our connection strings explicitly. Here, you only need to declare one reader, but you need to do a kind of branching logic, like if region is 'us-west-2' then use the 'us-west-2 environments
The rest of it will be handled by the library.
GitHub Repository
Sounds awesome? Exactly, now you can save your time more on your business logic. Let this library handle your connection database.
Find the GitHub repo here:
And feel free to raise an issue or PR if you have a better idea for improving this tiny yet useful library.
