|
| 1 | +<p align="center"> |
| 2 | + <img src="https://user-images.githubusercontent.com/38481289/172237414-023c0b04-c597-44b7-8b14-b5b0c382dc07.png" width='70%'> |
| 3 | +</p> |
| 4 | +<p align="center"> |
| 5 | +<a href='https://join.slack.com/t/readysetcommunity/shared_invite/zt-1c7bxdxo7-Y6KuoLfc1YWagLk3xHSrsw'><img src='https://img.shields.io/badge/Slack-Join%20the%20community-4A154B?style=flat&logo=slack&logoColor=white'></a> |
| 6 | +<a href='https://readysettech.typeform.com/to/BqovNk8A?typeform-source=github.com'><img src='https://img.shields.io/badge/ReadySet-Join%20the%20cloud%20waitlist-7648B5?logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABgAAAAYCAYAAADgdz34AAADrUlEQVRIS41V6UsVURQ/d2Z8vkVsoQ0iKsMgyd4rhGghogiqL0VQaUGLQQvtVPihsJ20KEgrxchK2vsg9CkikugPCNqQsiKwIqISKyzfmzmdu83ceU+rB8PcN3Pn/M75/X7nXAb/+HVsf78crMwNh3lg0+UwV909iDBYXVBb0vK3EKy/l483f/Zsy2W2xYMGgbNBbAJ0aE/iWLLPWDkPby1Fe9TQroyZMQ8gAhGQJcDClVAi4ln8SzrCmsrSZtIhgLZNvxdbmG7lWQaZyo9lJZIe+S4A1Wu+J2phuXM0dVOD+ABts9FJF/em+Sadbfa9tG44fHv0A761flVACtxMiL5vp0rKVCU+wJ11HvIMdbZBcJl52ZnBfuUvd7ynigKxw5WFNREAFzd7GCfm4i4qgDAVOoAGebH9IwnrQvGJkcAcBp27XyvKQtRhoiZpCYCzOwggA5AgkFhGgsw/74iMH276meUiWWXJ8aFgRy2/qs7dbwwLyz2FtaWMHazOVMR7rGsyOIH00t0FmLAEYdxCCx5s/JWjSeFYC8bvGugHRw/h3Z53ucLb7ipWtV/RQwAmSJRA+tNk6ukBRI2M7/UidFSRJkD0CIOETcK2HkHiH0FooC+qRKzp7outvF5WHQMnziAvweBtczf8fNbTp4W1bqyyFpFn7oNoLYyKNMjkLQ7kxVEEf3Lku19hqn6IqObLvW7outtlCO4CqziFGFPBOEjNAenchp0oNImTJtxdE8sBIgl+ITxv+BXiO2ThnZ0+AKeLLao3AYxKDMqSMxnEYwj5BQivbmb67AEO8nTbJ6VD0IBsXiPXQHJuVqKfjR4GMJwYKIgCfLhvatJXx3OhdQPKO5vVhCsjHlyROoRBEkRR0QiAKD3//ka6LKr6RHZ80M2BGQgEZDcz210vCJ9+QVYhbKqAEqRH3V4Gh/eRFlwH5SptZR5wQVYzZs+uSfUjmABItaBHAZgGCVUihJY21sElGAm/AmDMnOxm1FPXhSlnhkkAAXKZqlD+593M14EmUvzqowwadxFNHFC5S4xu3mBEiz6ceCXTzg0SsYPz4AA6qSJIa3HDmsgKuIVPVinKjNkVgEiR2732yIbscc3RJlzF8vwMXDf5DnW40fHh2YXBIWR7a+c2xi5pZnLP0Vtop3og49P1Fwsn0kxMXzkgEXq6rfxltxn9C379HvrJFj1CAncJ2gzR5Vq8xzVnrWB2/w+A3jOjGVeTTS/5mqiRLmYXepX7DzkXzYyz138Ac/pGks+x6G0AAAAASUVORK5CYII='></a> |
| 7 | +</p> |
| 8 | + |
| 9 | +--- |
| 10 | + |
| 11 | +ReadySet is a SQL caching engine designed to help developers enhance the performance and scalability of their existing database-backed applications. |
| 12 | + |
| 13 | +- [What is ReadySet?](#what-is-readyset) |
| 14 | +- [Getting Started](#getting-started) |
| 15 | +- [Documentation](#documentation) |
| 16 | +- [Join the Community](#join-the-community) |
| 17 | +- [Development](#development) |
| 18 | +- [License](#license) |
| 19 | + |
| 20 | +## What is ReadySet? |
| 21 | + |
| 22 | +ReadySet is a lightweight SQL caching engine that precomputes frequently-accessed query results and automatically keeps these results up-to-date over time as the underlying data in your database changes. ReadySet is wire-compatible with MySQL and Postgres and can be adopted without code changes. |
| 23 | + |
| 24 | +<br> |
| 25 | +<p align="center"> |
| 26 | + <img src='https://user-images.githubusercontent.com/38481289/172237407-e0546ef3-2095-49ab-be82-a177e507c6d1.png' width='70%'> |
| 27 | +</p> |
| 28 | +<br> |
| 29 | + |
| 30 | +ReadySet acts as both a SQL cache and a proxy– when you first connect ReadySet to your application, it defaults to proxying all of your queries to your backing database, so it doesn't change the behavior of your application. |
| 31 | + |
| 32 | +From there, you can monitor the performance of your queries using the ReadySet dashboard, and cache a subset of them in ReadySet. ReadySet automatically keeps cached results up-to-date over time as the underlying data in your database changes, so there's no need to write cache maintenance logic. |
| 33 | + |
| 34 | + |
| 35 | +## Getting Started |
| 36 | + |
| 37 | +### Deploy ReadySet |
| 38 | +The ReadySet orchestrator is a command line tool that uses Docker Compose to spin up a ReadySet instance on your local machine. The following command downloads and runs the ReadySet orchestrator: |
| 39 | + |
| 40 | +``` |
| 41 | +bash -c "$(curl -sSL https://launch.readyset.io)" |
| 42 | +``` |
| 43 | +The orchestrator will prompt you to select a wire protocol, and to connect ReadySet to an existing database (or create a new one). |
| 44 | + |
| 45 | +### Connect to ReadySet |
| 46 | + |
| 47 | +- **To connect to ReadySet via the MySQL or Postgres command line**, run the command that the orchestrator prints after successfully deploying a ReadySet instance. |
| 48 | +- **To connect ReadySet to your application**, use the ReadySet connection string (printed by the orchestrator after successfully deploying ReadySet) in the same way you would if it were a MySQL or Postgres connection string. Check out our [docs](https://docs.readyset.io/connecting) for more information. |
| 49 | + |
| 50 | +Once you connect to ReadySet, you can start issuing queries as you would with a traditional database. |
| 51 | + |
| 52 | +### Monitor Query Performance |
| 53 | + |
| 54 | +By default, all queries sent to ReadySet are proxied to the backing database. You can see the current list of proxied queries and their performance profiles via the ReadySet dashboard which is accessible on `http://localhost:4000`. |
| 55 | + |
| 56 | + |
| 57 | +### Cache Queries |
| 58 | +ReadySet supports caching both prepared statements (i.e. parameterized queries) and one-off queries. To cache a query in ReadySet, you’ll need either the query ID (which can be found in the dashboard, or by running `SHOW CACHES`) or the full query text (i.e. the `SELECT` statement). |
| 59 | + |
| 60 | +From there, you can run `CREATE CACHE FROM <query ID>` or `CREATE CACHE FROM <select statement>` via the MySQL or Postgres client. |
| 61 | + |
| 62 | +#### Write Handling |
| 63 | +You can either send writes to ReadySet or directly to your backing database. If you send ReadySet a write, it will be proxied to your backing database. ReadySet waits to receive updates from your database's bin logs before updating the cached state to reflect those writes. |
| 64 | + |
| 65 | +**Note:** The ReadySet Orchestrator is updated frequently with releases built off our internal ReadySet repository. Because of this, the orchestrator may have features not yet in our public repo. We're working on making our live codebase available on Github. |
| 66 | + |
| 67 | +## FAQs |
| 68 | +**Q: How does ReadySet work under the hood?** |
| 69 | + |
| 70 | +A: The heart of ReadySet is a query engine based on partially-stateful, streaming data flow. To learn more about how it works, see [our docs](https://docs.readyset.io/concepts/overview). |
| 71 | + |
| 72 | +**Q: How does ReadySet keep cached state up to date?** |
| 73 | + |
| 74 | +A: ReadySet receives updates about data changes from your backing database via binlog replication and uses those updates to automatically update its internal state. |
| 75 | + |
| 76 | +**Q: Do I have to send all of my database traffic to ReadySet?** |
| 77 | + |
| 78 | +A: You can if you want to, but it’s not required. You can manually route a subset of your traffic to ReadySet (as you would with a traditional database read replica), or you can send all of it to ReadySet. It's important to note that not all of the queries sent to ReadySet need to be cached– you have fine-grained control over what is cached vs. what is proxied. |
| 79 | + |
| 80 | +## Documentation |
| 81 | +For more information, check out our full docs site [here](http://docs.readyset.io). |
| 82 | + |
| 83 | +## Join the Community |
| 84 | + |
| 85 | +For questions or support, join us on the [ReadySet Community Slack](https://join.slack.com/t/readysetcommunity/shared_invite/zt-18z5bxv02-C5R08CYFL00jXmIqjd7Ziw). You can also post questions on our [Github forum](https://github.com/readysettech/readyset/discussions). Everyone is welcome! |
| 86 | + |
| 87 | +## Development |
| 88 | + |
| 89 | +### Prerequisites |
| 90 | + |
| 91 | +#### Install Dependencies |
| 92 | + |
| 93 | +Prior to running ReadySet, you may need to install the following dependencies: |
| 94 | +clang |
| 95 | +libclang-dev |
| 96 | +libssl-dev |
| 97 | +liblz4-dev |
| 98 | +build-essential |
| 99 | + |
| 100 | +**macOS:** |
| 101 | +``` |
| 102 | +brew install lz4 |
| 103 | + |
| 104 | +``` |
| 105 | + |
| 106 | +**Ubuntu:** |
| 107 | +``` |
| 108 | +sudo apt update && sudo apt install -y build-essential libssl-dev pkg-config llvm clang liblz4-dev cmake |
| 109 | +``` |
| 110 | +#### Install Rust |
| 111 | +ReadySet is written entirely in Rust. If you don’t already have Rust installed, you can install it via rustup (select the version of ‘nightly’ specified in the `rust-toolchain` file): |
| 112 | + |
| 113 | +```curl https://sh.rustup.rs -sSf | sh``` |
| 114 | +#### Runtime Dependencies: Upstream Database and Consul |
| 115 | +##### Local Development |
| 116 | +To streamline local development, all runtime dependencies (Consul, MySQL, Postgres) can be run with: |
| 117 | + |
| 118 | +``` |
| 119 | +cp docker-compose.override.yml.example docker-compose.override.yml |
| 120 | +docker-compose up -d |
| 121 | +``` |
| 122 | + |
| 123 | +##### Full Deployment |
| 124 | +ReadySet runs alongside a backing MySQL or Postgres database and uses Consul for leader election. You’ll need to pass in your database’s connection string when running the ReadySet server and adapter, and have a Consul instance running. |
| 125 | + |
| 126 | +*Note:* Consul persists data per deployment. Using the same deployment name will carry over any previously installed queries into a new deployment. |
| 127 | + |
| 128 | + |
| 129 | +### Run ReadySet |
| 130 | +#### ReadySet Server |
| 131 | + |
| 132 | +First, compile and run ReadySet server. |
| 133 | + |
| 134 | +``` |
| 135 | +cargo run --bin readyset-server --features display_literals --release -- --replication-url <upstream-url> --deployment <deployment name> |
| 136 | +``` |
| 137 | + |
| 138 | +If using the databases supplied by the docker-compose environment in this repository, replace <upstream-url> with the URL of the database corresponding to your database engine: |
| 139 | + |
| 140 | +* MySQL: `mysql://root:[email protected]/readyset` |
| 141 | +* PostgreSQL: `postgresql://root:[email protected]/readyset` |
| 142 | + |
| 143 | +If running with an existing external database, replace <upstream-url> with the connection string for that database. |
| 144 | + |
| 145 | +#### ReadySet MySQL or Postgres Adapter |
| 146 | + |
| 147 | +Then, run the adapter binary corresponding to your upstream database (MySQL or Postgres) The adapter will communicate with servers that have the same deployment name. |
| 148 | + |
| 149 | +**MySQL** |
| 150 | +``` |
| 151 | +cargo run --bin readyset-mysql --features display_literals --release -- --upstream-db-url mysql://root:[email protected]/readyset --allow-unauthenticated-connections |
| 152 | + --address 0.0.0.0:3307 --deployment <deployment name> --prometheus-metrics --query-log --query-log-ad-hoc |
| 153 | + ``` |
| 154 | + |
| 155 | +**Postgres** |
| 156 | +``` |
| 157 | +cargo run --bin readyset-psql --features display_literals --release -- --upstream-db-url postgresql://postgres:[email protected]/readyset --allow-unauthenticated-connections |
| 158 | + --address 0.0.0.0:5433 --deployment <deployment name> --prometheus-metrics --query-log --query-log-ad-hoc |
| 159 | +``` |
| 160 | + |
| 161 | +The adapter listens for connections at the address specified in the `address` flag. |
| 162 | + |
| 163 | +The `query-log` and `query-log-ad-hoc` flags ensure that queries are sent to the Prometheus client running in the adapter. |
| 164 | + |
| 165 | +The `prometheus-metrics` flag exposes an HTTP endpoint in the adapter to allow querying of metrics. This can be reached with an HTTP GET request to <adapter-address>:6034/prometheus (e.g., `curl -X GET 127.0.0.1:6034/prometheus`). |
| 166 | + |
| 167 | +### Testing |
| 168 | + |
| 169 | +To run tests for the project, run the following command: |
| 170 | + |
| 171 | +``` |
| 172 | +cargo test --skip integration_serial |
| 173 | +``` |
| 174 | +##### Testing Notes |
| 175 | + |
| 176 | +Certain tests cannot be run in parallel with others, and these tests are typically in files postfixed with _serial. Running the entire set of tests for a package, i.e. `cargo test -p readyset-server` may fail if serial tests are included. |
| 177 | + |
| 178 | +Running tests may require increasing file descriptor limits. You can do so by running `ulimit -Sn 65535`. |
| 179 | + |
| 180 | +### Performance |
| 181 | + |
| 182 | +When running ReadySet in a performance-critical setting, make sure you compile with the `--release` flag. |
| 183 | + |
| 184 | +**Note:** This repository contains a snapshot of the current state of the |
| 185 | +ReadySet internal repository. Our team is currently working on getting our live |
| 186 | +codebase, including all development history, ready to be hosted on GitHub, at |
| 187 | +which point we'll force push to this repository. |
| 188 | + |
| 189 | +## License |
| 190 | + |
| 191 | +ReadySet is licensed under the BSL 1.1 license, converting to the open-source Apache 2.0 license after 4 years. It is free to use on any number of nodes. The ReadySet team is hard at work getting the codebase ready to be hosted on Github. |
| 192 | + |
| 193 | +ReadySet is also available as a paid cloud service that can be deployed to your accounts or hosted entirely by ReadySet. You can get early access to our cloud product by signing up [here](https://readysettech.typeform.com/to/BqovNk8A). |
0 commit comments