Autoscaling Moodle stack for Postgres and MySQL databases
This work is mostly based on Paulo Teixeira’s work. It adds several configurable features and switches to a web stack with caching. It also uses Azure resources for Redis, ObjectFS and Databases.
After deploying, these templates will provide you with a new Moodle site with caching for speed and scaling frontends to handle PHP load. The filesystem behind it is mirrored for high availability and optionally backed up through Azure. Filesystem permissions and options have also been tuned to make Moodle more secure than a default install.
ProxySQL is a high performance, high availability, protocol aware proxy for MySQL and forks (like Percona Server and MariaDB). All the while getting the unlimited freedom that comes with a GPL license.
Its development is driven by the lack of open source proxies that provide high performance.
Source: sysown/proxysql: High-performance MySQL proxy with a GPL license.
What is mydumper? and Why?
- Parallelism (hence, speed) and performance (avoids expensive character set conversion routines, efficient code overall)
- Easier to manage output (separate files for tables, dump metadata, etc, easy to view/parse data)
- Consistency – maintains snapshot across all threads, provides accurate master and slave log positions, etc
- Manageability – supports PCRE for specifying database and tables inclusions and exclusions
Source: maxbube/mydumper: Official mydumper project
MySQL Cluster powers the subscriber databases of major communication services providers as well as next generation web, cloud, social and mobile applications. It is designed to deliver:
- Real-time, in-memory performance for both OLTP and analytics workloads
- Linear scale-out for both reads and writes
- 99.999% High Availability
- Transparent, cross-shard transactions and joins
- Update-Anywhere Geographic replication
- SQL or native NoSQL APIs
- All that while still providing full ACID transactions.
Understand some of the highlights of MySQL Cluster 7.4:
- 200 Million queries per minute
- Active-Active geographic replication with conflict detection and resolution
- 5x faster on-line maintenance activities
- Enhanced reporting for memory and database operations
Source: MySQL Cluster | Andrew Morgan on Databases
The QAN is a special dashboard which enables database administrators and application developers to analyze database queries over periods of time and find performance problems. QAN helps you optimize database performance by making sure that queries are executed as expected and within the shortest time possible. In case of problems, you can see which queries may be the cause and get detailed metrics for them.
Source: PMM Query Analytics
A collection of views, functions and procedures to help MySQL administrators get insight in to MySQL Database usage.
Source: mysql/mysql-sys: The MySQL sys schema
Recently, nginx 1.9 introduced support for TCP load balancing – similar to what HAProxy is capable of. The one major drawback is that it does not support advanced backend health checks. This is required when running MySQL Galera Cluster, as we’ll explain in the next section. Note that this limitation is removed in the paid-only edition called NGINX Plus.
מקור: nginx as Database Load Balancer for MySQL or MariaDB Galera Cluster | Severalnines
mytop (Monitor mysql process list with top like screen updates and adiditonal stats like cpu/io usage)
מקור: mytop (Monitor mysql process list with top like screen updates and adiditonal stats like cpu/io usage)
- Deploy, manage, monitor and scale entire clusters
- SQL and NoSQL cluster topologies
- SQL load balancing via HAProxy
ב-clustercontrol | Severalnines.
Vitess is available as an open-source project and runs best in a containerized environment. With Kubernetes and Google Container Engine as your container cluster manager, it’s now a lot easier to get started. We’ve created a single deployment configuration for Vitess that works on any platform that Kubernetes supports.
In addition to being easy to deploy in a container cluster, Vitess also takes full advantage of the benefits offered by a container cluster manager, in particular:
- Horizontal scaling – add capacity by launching additional nodes rather than making one huge node
- Dynamic placement – let the cluster manager schedule Vitess containers wherever it wants
- Declarative specification – describe your desired end state, and let the cluster manager create it
- Self-healing components – recover automatically from machine failures
In this environment, Vitess provides a MySQL storage layer with improved durability, scalability, and manageability.
ב-Google Cloud Platform Blog: Scaling MySQL in the cloud with Vitess and Kubernetes.