The recommended DBSyncer is an open source data synchronization middleware.
Project Description
DBSyncer is an open source data synchronization middleware. Provides synchronization scenarios such as Mysql, Oracle, SqlServer, Elasticsearch(ES), Kafka, and SQL(Mysql/Oracle/SqlServer). You can upload plug-ins to customize synchronous conversion services, monitor full and incremental data statistics, and warn application performance.
- Combination drive, custom library synchronization to library combination, combination between relational database and non-relational, any collocation table synchronization mapping relationship
- Real-time monitoring, driving full or incremental real-time synchronization of operating status, results, synchronization logs and system logs
- Develop plugins and customize conversion synchronization logic
Installation and configuration
< Steps
- Install JDK 1.8 (details omitted)
- Download installation package DBSyncer-1.0.0-Beta.zip
- Decompress the installation package, run bin/startup.bat for Windows, and bin/startup.sh for Linux.
- Open your browser to http://127.0.0.1:18686
- Account and password: admin/admin
Incremental Synchronization Configuration (source repository)
Mysql
- Dump Binlog binary logs. The Master synchronizes the Slave, creates IO threads to read data, writes to relaylog, and captures incremental data based on message subscriptions.
- Configuration
Modify the my.ini file
# Service unique ID
server_id=1
log-bin=mysql_bin
binlog-format=ROW
max_binlog_cache_size = 256M
max_binlog_size = 512M
expire_logs_days = 7
# Listen to synchronized libraries, multiple libraries using commas ", "concatenate
replicate-do-db=test
Oracle
- CDN Registration Subscription. Listen to add, delete and change events, get rowid, execute SQL query according to rowid, get the changed data.
- grants the account the permission to listen, and requires that the target source table must define a varchar field of length 18, and implement the operation by receiving the rowid value.
grant change notification to Your account
Timing
Assumed source table data format
Preview
Driver management
Driver details
Driver table field relationship configuration
Monitor