mirrors/zits
1
0 Fork 0
mirror of https://git.robbyzambito.me/zits synced 2026-11-05 15:51:40 +00:00
Code Issues Packages Projects Releases Wiki Activity
112 Commits 6 Branches 0 Tags
dev
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
zambyte 1ee0f9263c Properly handle large messages 
add max bytes setting
 2026-01-12 22:21:48 -05:00
zambyte c5ad98adc6 max buffer size way too small 2026-01-12 19:15:11 -05:00
zambyte 9f690fe27a Add a warning log when producers are stalled 2026-01-12 18:08:38 -05:00
zambyte 7bcc0c19aa Significant speed improvement 2026-01-12 09:37:47 -05:00
zambyte 72e7df5d5c Use appendAssumeCapacity instead of appendBounded catch unreachable 
Basically the same thing.
 2026-01-11 12:43:17 -05:00
zambyte 05ad1b8ffc Use cache line size for the cpu 2026-01-10 20:46:14 -05:00
zambyte 8a2bcebd60 Cache align hot buffers 2026-01-10 20:14:42 -05:00
zambyte a93a1f0906 Use much higher buffer sizes 2026-01-10 16:49:50 -05:00
zambyte 0861703ddc Sleep to go faster 
The problem was I was basically flushing twice for every message when
doing request reply.
This gives the sender the opportunity to finish writing a full message
to the queue, which we then check for before flushing.

This makes request reply latency benchmarks go down from like 90ms to
200us.
 2026-01-10 16:42:43 -05:00
zambyte 99ea755658 Send should be uncancelable 2026-01-10 11:37:47 -05:00
zambyte 78b23ee59c Use Subscription.send 2026-01-10 09:21:18 -05:00
zambyte ad13706d1b Properly handle disconnect 2026-01-10 09:21:18 -05:00
zambyte 4a228c8dba switch to uuids for clients 2026-01-10 09:21:18 -05:00
zambyte aec871ebdb Port to latest 0.16.0 
Use juicy main ;)
 2026-01-10 09:21:18 -05:00
zambyte 0ebc39b5e8 parsing cleanup 2026-01-10 00:12:52 -05:00
zambyte f4b545f852 Improve errors in parse API 2026-01-08 23:46:49 -05:00
zambyte ed99115969 More robust parsing and error propagation 2026-01-08 22:35:35 -05:00
zambyte d8488fde49 support hpub 
fixed issue where not all data was being sent
request reply has a performance issue but technically works
 2026-01-08 16:47:52 -05:00
zambyte 45feccbad8 WAY FASTER but doesn't send all? 
Seems to not flush the last message
 2026-01-07 23:19:19 -05:00
zambyte 96a3705069 starting zero alloc parsing 2026-01-07 23:19:19 -05:00
zambyte e2a60c9427 Rename to match actual subcommand 2026-01-06 23:11:48 -05:00
zambyte 3674792e3f scope parse logger 
also change internal name to match public name
 2026-01-06 23:04:19 -05:00
zambyte b6762ccb7c Cleaner SIGINT handling 
Use a Mutex to wait for the signal handler to fire instead of checking
an atomic boolean over and over again.
 2026-01-06 23:03:21 -05:00
zambyte 3b490fc3c8 Cleanup Server.zig 2026-01-06 22:27:13 -05:00
zambyte 4896928352 Major restructuring 
This makes things much easier to use as a library
 2026-01-06 21:59:41 -05:00
zambyte cc03631838 Better cancelation handling 
Based on this conversation with Andrew
https://ziggit.dev/t/am-i-canceling-my-std-io-group-incorrectly/13836
 2026-01-06 21:21:14 -05:00
zambyte b87412ee66 Restructuring 
Add a bunch of tests for the client
 2026-01-06 20:43:49 -05:00
zambyte 025a5344c8 Return error.Canceled from concurrent group task 2026-01-06 17:14:18 -05:00
zambyte c676a8218e Support queue groups 2026-01-06 14:06:22 -05:00
zambyte 81a93654a1 Don't reuse address 
This was a temporary workaround for when I was not cleanly exiting.
Now that I am, this is not necessary.
 2026-01-06 10:20:04 -05:00
zambyte 6e9f6998bd Use client allocator to own incoming messages to a client 2026-01-06 10:04:10 -05:00
zambyte 318d467f5c Optimize PUB and HPUB parsing 
This will try to take better advantage of the buffered reading.
Instead of pushing one byte at a time to the array list for each
section, find the end index for each section, then alloc the arraylist
and copy the data into it all at once.
 2026-01-05 20:13:24 -05:00
zambyte 3342aa22ab Update tests to work again 2026-01-05 18:26:43 -05:00
zambyte 1d2af4a69a Simplified queue access 
Also correctly move resetting the task to the end instead of defer.
We don't want to reset the task in the case of an error, so shouldn't
use defer.
 2026-01-05 13:56:40 -05:00
zambyte 80d14f7303 Display help when there is no subcommand 2026-01-05 13:47:27 -05:00
zambyte e50d53ee7e Add Payload type 
stores short message buffers in a colocated array, overflowing to an
allocated slice when needed.
 2026-01-05 10:34:31 -05:00
zambyte ca43a12b9b Using separate queue for high throughput messages 2026-01-04 23:36:44 -05:00
zambyte 69528a1b72 Probe for optimal network buffer size. 
We want to match the underlying system socket buffer.
Filling this buffer minimizes the number of syscalls we do.
Larger would be a waste.

Also changed parser to use enums that more closely match the NATS
message types.
 2026-01-04 20:57:31 -05:00
zambyte e81bcda920 holy moly goes way fast!!! 
like 150 mbps now
 2026-01-03 06:08:40 +00:00
zambyte fbc137e2b3 Kill dead code and use higher buffer 2026-01-03 06:02:47 +00:00
zambyte dcd09e2f10 cleanup imports 2026-01-03 05:54:14 +00:00
zambyte bd9829f684 Organize things 
Making it easier to use the server as a library
 2026-01-03 05:33:56 +00:00
zambyte a4ec798521 Fix parse errors, ownership errors. 2026-01-03 03:17:13 +00:00
zambyte 9e32d014c2 Restructuring parser 
Adding tests fore everything
 2026-01-03 02:34:04 +00:00
zambyte f99b44fdb2 Fix double free 
was freeing the wrong element before.
 2026-01-03 02:33:12 +00:00
zambyte 5a7d3caf9c Subject validation 2026-01-02 23:57:40 +00:00
zambyte a21dbfe3bb Remove unnecessary explicit enum 
This can be computed (as it is now)
 2026-01-02 23:31:16 +00:00
zambyte 0f851a140d Fix possible race condition 
since the queue was being set in an async task and we were then calling send asserting that the queue was set, we could have triggered a panic.
didn't run into it but seemed likely to cause issues in the future.

also compute the buffer size for operators at comptime.
 2026-01-02 23:13:54 +00:00
zambyte 67908cf198 Move handshake from client to server 2026-01-02 22:37:54 +00:00
zambyte 29e5b92ee0 Only check for ctrl+c every 10 ms 2026-01-02 20:50:57 +00:00
zambyte 90b5b1548f Add branch hints for high performance messaging. 2026-01-02 20:35:58 +00:00
zambyte 9e9f7e8e50 Fix connect parsing 2026-01-02 20:28:26 +00:00
zambyte 5f24108014 Don't assume capacity 2026-01-02 19:53:53 +00:00
zambyte 6374952941 remove debug log 2026-01-02 19:34:59 +00:00
zambyte c38d13e911 Rework pub and hpub parse 
support hpub in general, and properly support reply subjects
 2026-01-02 19:16:07 +00:00
zambyte cd5281030e 2026-01-02 18:10:25 +00:00
zambyte 2be370e379 Support subject patterns 
clean up some tests
 2026-01-02 15:29:58 +00:00
zambyte 539255adb1 broke 100 mbps!! 
coder@08714a4174bb:~$ nats bench sub foo -s localhost:4223
14:33:02 Starting Core NATS subscriber benchmark [clients=1, msg-size=128 B, msgs=100,000, multi-subject=false, subject=foo]
14:33:02 [1] Starting Core NATS subscriber, expecting 100,000 messages
Finished      0s [===============================================================================================================================] 100%

NATS Core NATS subscriber stats: 934,205 msgs/sec ~ 114 MiB/sec ~ 1.07us
 2026-01-02 14:34:32 +00:00
zambyte faa552969d Nicer logging 2026-01-02 14:31:34 +00:00
zambyte 810e5f8211 Much better cleanup, still fast B) 2026-01-02 14:11:23 +00:00
zambyte aca265f095 Cleanup on error 2026-01-02 13:58:23 +00:00
zambyte 4bf064056c reorganize but crashing 
not sure why, seems like i'm using the right allocators everywhere?
need to take another pass at this later.
 2026-01-02 04:01:03 +00:00
zambyte a71f08e1f6 formatting 2026-01-02 02:18:37 +00:00
zambyte 9ee8317cb0 Remove dead comments 2026-01-01 22:14:36 +00:00
zambyte 5a395bdadf Use group to make it easier to clean up client tasks on exit 2026-01-01 22:14:36 +00:00
zambyte a3e026ebf1 Cleanup for deinitializing 2026-01-01 22:14:36 +00:00
zambyte fc68749669 gracefully exit 
simplify code
clean up dead code
 2026-01-01 20:26:41 +00:00
zambyte 987dc492a6 97 mbps !!! super fast 
dosen't flush every message, pulls batches from the queue to send, and flushes at the end of each batch.
batches are a min of 1 message, but may be more.
 2026-01-01 19:16:58 +00:00
zambyte 86558986ef Slower, but probably more correct 2026-01-01 18:50:52 +00:00
zambyte ca95c9a06c Lock subs when reading it 
should probably make a copy instead of doing io in the mutex
 2026-01-01 18:50:03 +00:00
zambyte ec1912ba7e Use a small stack buffer for client io 2026-01-01 16:55:04 +00:00
zambyte 7baa2ed2ec Use master yazap 
Works against the latest master branch for zig again :)
 2026-01-01 16:29:29 +00:00
zambyte 45bd63dbe1 Actually fast again??? 
way faster than before even??

coder@08714a4174bb:~$ nats bench pub foo -s localhost:4223
05:12:23 Starting Core NATS publisher benchmark [clients=1, msg-size=128 B, msgs=100,000, multi-subject=false, multi-subject-max=100,000, sleep=0s, subject=foo]
05:12:23 [1] Starting Core NATS publisher, publishing 100,000 messages
Finished      0s [====================================================================================] 100%

NATS Core NATS publisher stats: 574,666 msgs/sec ~ 70 MiB/sec ~ 1.74us

So cool.

src/server/client.zig JJ: M src/server/main.zig JJ: JJ: Lines starting with "JJ:" (like this one) will be 
removed.
 2026-01-01 05:22:03 +00:00
zambyte 5dea33367e reorganize some things 2026-01-01 05:22:03 +00:00
zambyte f289ab7893 Rename ClientState to Client 2026-01-01 05:22:03 +00:00
zambyte 4fcb9e3943 Heap allocate client buffers 2026-01-01 05:22:02 +00:00
zambyte 0233bc278c Avoid queues completely 
This is quite slow
 2026-01-01 05:22:02 +00:00
zambyte b447883d10 Reorganized things 2026-01-01 05:22:02 +00:00
zambyte 7af7a30ed2 things are running quite smoothly!!! 
coder@08714a4174bb:~$ nats bench sub foo -s localhost:4223
03:28:04 Starting Core NATS subscriber benchmark [clients=1, msg-size=128 B, msgs=100,000, multi-subject=false, subject=foo]
03:28:04 [1] Starting Core NATS subscriber, expecting 100,000 messages
Finished      6s [====================================================================================] 100%

NATS Core NATS subscriber stats: 14,691 msgs/sec ~ 1.8 MiB/sec ~ 68.06us
 2026-01-01 05:22:02 +00:00
zambyte a781876038 Simplified a ton of things and cleaned up ownership 2026-01-01 05:22:02 +00:00
zambyte 0f138e5984 some cleanup and freeing 2026-01-01 05:22:02 +00:00
zambyte e60a566a7c Cleanup client 
Break up creating and starting the client process.
I think this should simplify storing the std.Io.Queue on the stack.
Before I was storing it on the heap because it was hard to make it point to the same location if I was initializing the client on the stack.
 2026-01-01 05:22:02 +00:00
zambyte 1cbd030037 Rename assert to expect 
Assert implies panic, expect implies error
 2026-01-01 05:22:02 +00:00
zambyte 1b6447a986 Fix sub parse bug 
Was accidentally consuming one more byte than I was expecting when reaching the end of the second term.
This was causing the parser to work properly in the case that a queue group was specified, but failing and consuming the next message (usually a PING) as the SID.
 2025-12-31 00:56:34 +00:00
zambyte d4566eba52 Reuse port 
This lets me dev cycle faster
Shouldn't have to do this though, should be cleaning up properly
 2025-12-31 00:55:38 +00:00
zambyte fe4c1d410b More cleanup 2025-12-29 04:56:54 +00:00
zambyte b9d0672971 Parsing cleanup 2025-12-29 04:49:25 +00:00
zambyte 4bf5ddca15 publish works 
starting to use errors instead of unreachable for stream parsing
 2025-12-29 04:23:16 +00:00
zambyte 335c4aa092 2025-12-29 00:35:06 +00:00
zambyte 18f4475aaf 2025-12-28 22:22:33 +00:00
zambyte 50870da1d9 made some progress on subscriptions 2025-12-10 00:11:12 -05:00
zambyte 5fd580045d Add branch hint for message publish 2025-12-09 21:16:44 -05:00
zambyte 45e8c4cc08 sending errors on pub sub! 2025-12-09 18:22:41 -05:00
zambyte 30cc017081 The issue was the writer is not working to send pong ?? 2025-12-09 17:56:07 -05:00
zambyte 1eeb55ff4d Made progress, but not perfect. 
the message isn't crossing container boundaries, but it works in the
test!
 2025-12-08 21:09:16 -05:00
zambyte 826da348a5 2025-12-06 19:45:05 -05:00
zambyte 4afdf32beb 2025-12-02 22:03:17 -05:00
zambyte aceb671ddc 2025-12-02 19:53:03 -05:00
zambyte 41f4ee721b Switch from clap to yazap 
This is compatible with the latest 0.16.0 nightly build.
It is also a bit less magic than clap.
 2025-11-29 22:46:29 -05:00
zambyte bd9ed88e5c 2025-11-29 18:30:39 -05:00