In my last post, I mentioned that Regex in C# is very slow. Today we’ll look at the performance differences between Regex, Substring, and Slice in C#. In order to be consistent, I am making use of BenchmarkDotNet. I wanted to run these tests on both .NET Core 2.1 and .NET 4.7.2 to get a comparison between them as well. However this posed a small problem, as the APIs for .NET and .NET Core differ slightly for the Span<T> and Memory<T> types, specifically when it comes to parsing int values from these types. This required me to run the benchmarks separately, which I did by targeting both frameworks and utilizing conditional compilation blocks.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
BenchmarkDotNet=v0.11.1, OS=Windows 10.0.17134.228 (1803/April2018Update/Redstone4)
Intel Core i7-6700K CPU 4.00GHz (Max: 2.00GHz) (Skylake), 1 CPU, 8 logical and 4 physical cores
Frequency=3914064 Hz, Resolution=255.4889 ns, Timer=TSC
.NET Core SDK=2.1.400
[Host] : .NET Core 2.1.2 (CoreCLR 4.6.26628.05, CoreFX 4.6.26629.01), 64bit RyuJIT
Core : .NET Core 2.1.2 (CoreCLR 4.6.26628.05, CoreFX 4.6.26629.01), 64bit RyuJIT
Job=Core Runtime=Core
Method | Mean | Error | StdDev | Rank | Gen 0 | Allocated |
------------ |-----------:|----------:|----------:|-----:|-------:|----------:|
OneRegex | 2,086.5 ns | 38.576 ns | 36.084 ns | 4 | 0.2174 | 920 B |
FourRegexes | 2,937.0 ns | 57.177 ns | 56.155 ns | 5 | 0.4082 | 1720 B |
Substring | 601.7 ns | 5.830 ns | 5.453 ns | 1 | 0.1440 | 608 B |
Slice | 778.7 ns | 13.601 ns | 12.723 ns | 2 | - | 0 B |
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
|
BenchmarkDotNet=v0.11.1, OS=Windows 10.0.17134.228 (1803/April2018Update/Redstone4)
Intel Core i7-6700K CPU 4.00GHz (Skylake), 1 CPU, 8 logical and 4 physical cores
Frequency=3914064 Hz, Resolution=255.4889 ns, Timer=TSC
[Host] : .NET Framework 4.7.2 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.3132.0
Clr : .NET Framework 4.7.2 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.3132.0
Job=Clr Runtime=Clr
Method | Mean | Error | StdDev | Rank | Gen 0 | Allocated |
------------ |-----------:|----------:|----------:|-----:|-------:|----------:|
OneRegex | 1,901.1 ns | 28.280 ns | 23.615 ns | 4 | 0.2174 | 920 B |
FourRegexes | 2,653.1 ns | 37.611 ns | 35.181 ns | 5 | 0.4082 | 1720 B |
Substring | 610.6 ns | 7.735 ns | 7.235 ns | 1 | 0.1497 | 632 B |
Slice | 745.2 ns | 8.718 ns | 7.728 ns | 2 | 0.0410 | 176 B |
|
What I find interesting about these results is the comparison of Substring and Slice. In my experience, Slice is considerably faster than Substring, but this is most likely due to the number of strings I manipulate while parsing strings. The more Substring calls there are, the more garbage is generated. This requires more garbage collection.
Looking a bit more into the results, we can see that, even though Slice results in 0 allocations in .NET Core, the full framework is still a bit faster. The reason for 0 allocations in .NET Core is due to native API support for parsing ReadOnlyMemory<char> in the int, and other primitive, types. The curiosity here is that, despite needing to call ToString() on the slices before parsing the values, the .NET 4.7.2 CLR is still faster.
Lets see if we can replicate some of my real world experience in these tests and actually see what it takes. I rewrote the tests to use an actual mongod log line, and ran significantly more iterations. This puts more pressure on the GC as well as the parsers. This should amplify small variations allowing us to better see differences.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
BenchmarkDotNet=v0.11.1, OS=Windows 10.0.17134.228 (1803/April2018Update/Redstone4)
Intel Core i7-6700K CPU 4.00GHz (Skylake), 1 CPU, 8 logical and 4 physical cores
Frequency=3914067 Hz, Resolution=255.4887 ns, Timer=TSC
.NET Core SDK=2.1.500-preview-009297
[Host] : .NET Core 2.1.3 (CoreCLR 4.6.26725.06, CoreFX 4.6.26725.05), 64bit RyuJIT
Core : .NET Core 2.1.3 (CoreCLR 4.6.26725.06, CoreFX 4.6.26725.05), 64bit RyuJIT
Job=Core Runtime=Core
Method | Mean | Error | StdDev | Rank | Gen 0 | Gen 1 | Gen 2 | Allocated |
---------------- |------------:|-----------:|-----------:|-----:|-----------:|----------:|---------:|----------:|
OneRegex | 549.28 ms | 9.4628 ms | 8.3885 ms | 3 | 14000.0000 | 5000.0000 | - | 89.75 MB |
MultipleRegexes | 3,187.95 ms | 37.3505 ms | 31.1894 ms | 4 | 30000.0000 | 8000.0000 | - | 185.85 MB |
Substring | 127.87 ms | 0.6544 ms | 0.5465 ms | 2 | 29200.0000 | 7400.0000 | 200.0000 | 175.91 MB |
Slice | 91.64 ms | 1.2612 ms | 1.1797 ms | 1 | 2833.3333 | 1333.3333 | 166.6667 | 18.65 MB |
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
|
BenchmarkDotNet=v0.11.1, OS=Windows 10.0.17134.228 (1803/April2018Update/Redstone4)
Intel Core i7-6700K CPU 4.00GHz (Skylake), 1 CPU, 8 logical and 4 physical cores
Frequency=3914067 Hz, Resolution=255.4887 ns, Timer=TSC
[Host] : .NET Framework 4.7.2 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.3132.0
Clr : .NET Framework 4.7.2 (CLR 4.0.30319.42000), 64bit RyuJIT-v4.7.3132.0
Job=Clr Runtime=Clr
Method | Mean | Error | StdDev | Rank | Gen 0 | Gen 1 | Allocated |
---------------- |-----------:|-----------:|-----------:|-----:|-----------:|----------:|----------:|
OneRegex | 484.8 ms | 4.0251 ms | 3.7651 ms | 3 | 15000.0000 | 4000.0000 | 91.18 MB |
MultipleRegexes | 3,128.3 ms | 58.9805 ms | 57.9267 ms | 4 | 31000.0000 | 8000.0000 | 187.29 MB |
Substring | 178.1 ms | 1.3034 ms | 1.2192 ms | 2 | 29000.0000 | 7333.3333 | 177.34 MB |
Slice | 112.4 ms | 0.9331 ms | 0.8271 ms | 1 | 5000.0000 | 1600.0000 | 32.38 MB |
|
Well this is starting to show what I experienced while writing my log parser. Instead of using just random text, I took actual mongod log lines, obfuscated the information and parsed these. Now we can see just how big of a lead Slice can have over both Regex and Substring. This is both in terms of overall speed and in memory allocations, which can be significant.
When processing log files in excess of 1 GB, memory allocations and subsequent garbage collections can add a significant amount of time to total execution of the program.
This is by no means an end-all be-all test of these functions, and as always, your mileage may vary. It is important to not start by optimizing your code. Even if you are doing thousands of these operations, the difference between Substring and Slice will probably be minimal. While Slice can be very powerful, the support is currently fairly limited.
As always, I hope this helps and happy programming.
