Posts

Let’s first look at the following code snippet: package main import ( "fmt" "os" "runtime" "time" ) func main() { runtime.GOMAXPROCS(2) a := make(map[int]int) go func() { i := 0 for { a[1] = i i++ time.Sleep(1000) } }() for { if a[1] > 1000000 { fmt.Println(a[1]) os.Exit(1) } } } After compiling and running it (assuming your machine has 2 or more cores), you will get the following error: fatal error: concurrent map read and map write goroutine 1 [running]: runtime.throw(0x10c3e05, 0x21) /usr/local/Cellar/go/1.10.3/libexec/src/runtime/panic.go:616 +0x81 fp=0xc42004bf00 sp=0xc42004bee0 pc=0x10263f1 runtime.mapaccess1_fast64(0x10a5b60, 0xc42007e180, 0x1, 0xc42008e048) /usr/local/Cellar/go/1.10.3/libexec/src/runtime/hashmap_fast.go:101 +0x197 fp=0xc42004bf28 sp=0xc42004bf00 pc=0x1008d27 main.main() /Users/yusp/test/panic3/main.go:22 +0x7c fp=0xc42004bf88 sp=0xc42004bf28 pc=0x108e28c runtime.main() /usr/local/Cellar/go/1.10.3/libexec/src/runtime/proc.go:198 +0x212 fp=0xc42004bfe0 sp=0xc42004bf88 pc=0x1027c62 runtime.goexit() /usr/local/Cellar/go/1.10.3/libexec/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc42004bfe8 sp=0xc42004bfe0 pc=0x104e501 goroutine 5 [runnable]: time.Sleep(0x3e8) /usr/local/Cellar/go/1.10.3/libexec/src/runtime/time.go:102 +0x166 main.main.func1(0xc42007e180) /Users/yusp/test/panic3/main.go:18 +0x61 created by main.main /Users/yusp/test/panic3/main.go:13 +0x59 It seems straightforward; Golang’s map is not thread-safe, and concurrent read and write cause a panic. However, look at the error information on line /Users/yusp/test/panic3/main.go:18 +0x61, which points to line 18 of main.go where Sleep is called, not the actual point of concurrency issue. In a vast stack trace, it becomes even harder to locate the problem. ...

In a previous article “TiDB Source Code Reading Series (4) Overview of INSERT Statement”, we introduced the general process of the INSERT statement. Why write a separate article for INSERT? Because in TiDB, simply inserting a piece of data is the simplest and most common case. It becomes more complex when defining various behaviors within the INSERT statement, such as how to handle situations with Unique Key conflicts: Should we return an error? Ignore the current data insertion? Or overwrite existing data? Therefore, this article will continue to delve into the INSERT statement. ...

Installing HAProxy yum install haproxy is effective for CentOS 7. After installation, you can start the service using systemctl start haproxy. But don’t rush yet. Configuring HAProxy Add the following content to /etc/haproxy/haproxy.cfg. global # The content of global is generally fixed and quite understandable. log 127.0.0.1 local2 maxconn 4096 user haproxy group haproxy chroot /var/lib/haproxy daemon pidfile /var/run/haproxy.pid stats socket /var/run/haproxy.sock # Create a socket file for haproxy nbproc 40 # Start 40 processes to forward concurrently, higher versions can use nbthread, a threaded approach. defaults # This section is mostly copied, not entirely clear on the options. log global mode http option tcplog option dontlognull retries 3 option redispatch maxconn 1024 timeout connect 5000ms timeout client 50000ms timeout server 50000ms listen cdb_cluster 0.0.0.0:3030 # The actual proxy name and address for receiving services. ## cdb balance leastconn - the cluster listening on port 3030. mode tcp balance leastconn # This method is most suitable for databases; do not change. server cdb1 172.16.30.3:26257 check # Check seems to require a port for feedback status; without it, it might not work, but it doesn't matter. server cdb2 172.16.30.3:26258 check server cdb3 172.16.30.3:26259 check server cdb4 172.16.30.3:26260 check Start and Connect systemctl start haproxy to start the service. ...

Why Test TPC-C First of all, TPC-C is the de facto OLTP benchmark standard. It is a set of specifications, and any database can publish its test results under this standard, so there’s no issue of quarreling over the testing tools used. Secondly, TPC-C is closer to real-world scenarios as it includes a transaction model within it. In the flow of this transaction model, there are both high-frequency simple transaction statements and low-frequency inventory query statements. Therefore, it tests the database more comprehensively and practically. ...

Compiling Sysbench with pgsql Support CockroachDB uses the PostgreSQL protocol. If you want to use Sysbench for testing, you need to enable pg protocol support in Sysbench. Sysbench already supports the pg protocol, but it is not enabled by default during compilation. You can configure it with the following command: ./configure --with-pgsql Of course, preliminary work involves downloading the Sysbench source code and installing the necessary PostgreSQL header files required for compilation (you can use yum or sudo to install them). ...

Introduction to OLTP-Bench OLTP-Bench is an open-source benchmarking tool platform for OLTP scenarios from CMU’s DB Group. It was designed to provide a simple, easy-to-use, and extensible testing platform. It connects to databases via the JDBC interface, supporting the following test suites: TPC-C Wikipedia Synthetic Resource Stresser Twitter Epinions.com TATP AuctionMark SEATS YCSB JPAB (Hibernate) CH-benCHmark Voter (Japanese “American Idol”) SIBench (Snapshot Isolation) SmallBank LinkBench Detailed project information can be found here, and the GitHub page is here. ...

Background The English font set in Microsoft’s VS Code is monospaced, while the Chinese font is not. Method Download and install the fonts from here. This font combines Source Han Sans and is said to align punctuation strictly. Use the method described here to set up VS Code. PS: It is indeed aligned, although the English text appears a bit narrow.

Background The DDL paper on F1 serves as the foundation for TiDB’s DDL implementation. There are two main papers on F1: one provides an overview of F1’s DDL, and the other specifically details the schema change method for DDL. I personally believe the second is key and more confusing to me. There is an introduction to the second paper here, which can help in understanding. Understanding Online DDL Concept The DDL discussed here refers to online DDL. The concept of online DDL originates from databases like MySQL, whereas PostgreSQL and similar databases might not support it. This concept is also quite vague; the distinction is whether you need to use exclusive locks during DDL operations to block transactions. Therefore, all databases can perform online DDL; it just depends on whether they’re willing to put in the effort. For traditional businesses where 24/7 availability isn’t a priority, DDL operations can be performed during maintenance times or late at night. Even if a few users are online, at most, they might experience minor delays. However, modern internet businesses are strict about maintenance windows, creating a higher demand for non-blocking DDL, which MySQL, as a quintessential internet database, was first to support. The typical implementation involves creating a copy of the schema table, with operations being sent to both the new and old tables during the transition. ...

Background The CAP theorem has become one of the hottest theorems in recent years; when discussing distributed systems, CAP is inevitably mentioned. However, I feel that I haven’t thoroughly understood it, so I wanted to write a blog post to record my understanding. I will update the content as I gain new insights. Understanding I read the first part of this paper. The CAP theorem [Bre12] says that you can only have two of the three desirable properties of: ...

Background Travis CI is generally used for automating tests without needing to update the repository with the test outputs. This article explains how to automatically commit the results from Travis CI. Process The basic process references this gist. Below is the .travis.yml from the gist. language: ruby rvm: - 2.0.0 env: global: - USER="username" - EMAIL="[email protected]" - REPO="name of target repo" - FILES="README.md foo.txt bar.txt" - GH_REPO="github.com/${USER}/${REPO}.git" - secure: "put travis gem output here => http://docs.travis-ci.com/user/encryption-keys/" script: - ruby test.rb after_success: - MESSAGE=$(git log --format=%B -n 1 $TRAVIS_COMMIT) - git clone git://${GH_REPO} - mv -f ${FILES} ${REPO} - cd ${REPO} - git remote - git config user.email ${EMAIL} - git config user.name ${USER} - git add ${FILES} - git commit -m "${MESSAGE}" - git push "https://${GH_TOKEN}@${GH_REPO}" master > /dev/null 2>&1 Note here that MESSAGE should be quoted when committing, which the original gist did not include. ...