Posts

Background When I first started writing Golang, I was always looking for a convenient debugging tool. Back then, I came across documentation about using gdb to debug and also tried delve, but neither felt easy to use. Later, on someone’s advice, I went back to the good old print statements… Over the past couple of days, I was debugging go test and found that tests would always hang when run per package. I couldn’t think of a suitable method at first, so I thought of delve again. After giving it a try, I found it has become much more mature than before. ...

There are multiple versions of the Shadowsocks server side implementation. The original version was written in Python, and later, enthusiasts implemented it in various programming languages of their liking. Among all these implementations, I personally think the most reliable and stable one is the original Python version. The reason is simple - it has the most users. The Golang version is said to have the most features and also performs very well, making it quite powerful. This might be due to Golang’s inherent high performance and ease of implementation. There’s also an implementation using libev, a pure C implementation, which also offers good performance and is very lightweight. ...

Let’s first look at the following code snippet: package main import ( "fmt" "os" "runtime" "time" ) func main() { runtime.GOMAXPROCS(2) a := make(map[int]int) go func() { i := 0 for { a[1] = i i++ time.Sleep(1000) } }() for { if a[1] > 1000000 { fmt.Println(a[1]) os.Exit(1) } } } After compiling and running it (assuming your machine has 2 or more cores), you will get the following error: fatal error: concurrent map read and map write goroutine 1 [running]: runtime.throw(0x10c3e05, 0x21) /usr/local/Cellar/go/1.10.3/libexec/src/runtime/panic.go:616 +0x81 fp=0xc42004bf00 sp=0xc42004bee0 pc=0x10263f1 runtime.mapaccess1_fast64(0x10a5b60, 0xc42007e180, 0x1, 0xc42008e048) /usr/local/Cellar/go/1.10.3/libexec/src/runtime/hashmap_fast.go:101 +0x197 fp=0xc42004bf28 sp=0xc42004bf00 pc=0x1008d27 main.main() /Users/yusp/test/panic3/main.go:22 +0x7c fp=0xc42004bf88 sp=0xc42004bf28 pc=0x108e28c runtime.main() /usr/local/Cellar/go/1.10.3/libexec/src/runtime/proc.go:198 +0x212 fp=0xc42004bfe0 sp=0xc42004bf88 pc=0x1027c62 runtime.goexit() /usr/local/Cellar/go/1.10.3/libexec/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc42004bfe8 sp=0xc42004bfe0 pc=0x104e501 goroutine 5 [runnable]: time.Sleep(0x3e8) /usr/local/Cellar/go/1.10.3/libexec/src/runtime/time.go:102 +0x166 main.main.func1(0xc42007e180) /Users/yusp/test/panic3/main.go:18 +0x61 created by main.main /Users/yusp/test/panic3/main.go:13 +0x59 It seems straightforward; Golang’s map is not thread-safe, and concurrent read and write cause a panic. However, look at the error information on line /Users/yusp/test/panic3/main.go:18 +0x61, which points to line 18 of main.go where Sleep is called, not the actual point of concurrency issue. In a vast stack trace, it becomes even harder to locate the problem. ...

In a previous article “TiDB Source Code Reading Series (4) Overview of INSERT Statement”, we introduced the general process of the INSERT statement. Why write a separate article for INSERT? Because in TiDB, simply inserting a piece of data is the simplest and most common case. It becomes more complex when defining various behaviors within the INSERT statement, such as how to handle situations with Unique Key conflicts: Should we return an error? Ignore the current data insertion? Or overwrite existing data? Therefore, this article will continue to delve into the INSERT statement. ...

Installing HAProxy yum install haproxy is effective for CentOS 7. After installation, you can start the service using systemctl start haproxy. But don’t rush yet. Configuring HAProxy Add the following content to /etc/haproxy/haproxy.cfg. global # The content of global is generally fixed and quite understandable. log 127.0.0.1 local2 maxconn 4096 user haproxy group haproxy chroot /var/lib/haproxy daemon pidfile /var/run/haproxy.pid stats socket /var/run/haproxy.sock # Create a socket file for haproxy nbproc 40 # Start 40 processes to forward concurrently, higher versions can use nbthread, a threaded approach. defaults # This section is mostly copied, not entirely clear on the options. log global mode http option tcplog option dontlognull retries 3 option redispatch maxconn 1024 timeout connect 5000ms timeout client 50000ms timeout server 50000ms listen cdb_cluster 0.0.0.0:3030 # The actual proxy name and address for receiving services. ## cdb balance leastconn - the cluster listening on port 3030. mode tcp balance leastconn # This method is most suitable for databases; do not change. server cdb1 172.16.30.3:26257 check # Check seems to require a port for feedback status; without it, it might not work, but it doesn't matter. server cdb2 172.16.30.3:26258 check server cdb3 172.16.30.3:26259 check server cdb4 172.16.30.3:26260 check Start and Connect systemctl start haproxy to start the service. ...

Why Test TPC-C First of all, TPC-C is the de facto OLTP benchmark standard. It is a set of specifications, and any database can publish its test results under this standard, so there’s no issue of quarreling over the testing tools used. Secondly, TPC-C is closer to real-world scenarios as it includes a transaction model within it. In the flow of this transaction model, there are both high-frequency simple transaction statements and low-frequency inventory query statements. Therefore, it tests the database more comprehensively and practically. ...

Compiling Sysbench with pgsql Support CockroachDB uses the PostgreSQL protocol. If you want to use Sysbench for testing, you need to enable pg protocol support in Sysbench. Sysbench already supports the pg protocol, but it is not enabled by default during compilation. You can configure it with the following command: ./configure --with-pgsql Of course, preliminary work involves downloading the Sysbench source code and installing the necessary PostgreSQL header files required for compilation (you can use yum or sudo to install them). ...

Introduction to OLTP-Bench OLTP-Bench is an open-source benchmarking tool platform for OLTP scenarios from CMU’s DB Group. It was designed to provide a simple, easy-to-use, and extensible testing platform. It connects to databases via the JDBC interface, supporting the following test suites: TPC-C Wikipedia Synthetic Resource Stresser Twitter Epinions.com TATP AuctionMark SEATS YCSB JPAB (Hibernate) CH-benCHmark Voter (Japanese “American Idol”) SIBench (Snapshot Isolation) SmallBank LinkBench Detailed project information can be found here, and the GitHub page is here. ...

Background The English font set in Microsoft’s VS Code is monospaced, while the Chinese font is not. Method Download and install the fonts from here. This font combines Source Han Sans and is said to align punctuation strictly. Use the method described here to set up VS Code. PS: It is indeed aligned, although the English text appears a bit narrow.

Background The DDL paper on F1 serves as the foundation for TiDB’s DDL implementation. There are two main papers on F1: one provides an overview of F1’s DDL, and the other specifically details the schema change method for DDL. I personally believe the second is key and more confusing to me. There is an introduction to the second paper here, which can help in understanding. Understanding Online DDL Concept The DDL discussed here refers to online DDL. The concept of online DDL originates from databases like MySQL, whereas PostgreSQL and similar databases might not support it. This concept is also quite vague; the distinction is whether you need to use exclusive locks during DDL operations to block transactions. Therefore, all databases can perform online DDL; it just depends on whether they’re willing to put in the effort. For traditional businesses where 24/7 availability isn’t a priority, DDL operations can be performed during maintenance times or late at night. Even if a few users are online, at most, they might experience minor delays. However, modern internet businesses are strict about maintenance windows, creating a higher demand for non-blocking DDL, which MySQL, as a quintessential internet database, was first to support. The typical implementation involves creating a copy of the schema table, with operations being sent to both the new and old tables during the transition. ...