Chris Lu
02773a6107
Accumulated changes for message queue ( #6600 )
...
* rename
* set agent address
* refactor
* add agent sub
* pub messages
* grpc new client
* can publish records via agent
* send init message with session id
* fmt
* check cancelled request while waiting
* use sessionId
* handle possible nil stream
* subscriber process messages
* separate debug port
* use atomic int64
* less logs
* minor
* skip io.EOF
* rename
* remove unused
* use saved offsets
* do not reuse session, since always session id is new after restart
remove last active ts from SessionEntry
* simplify printing
* purge unused
* just proxy the subscription, skipping the session step
* adjust offset types
* subscribe offset type and possible value
* start after the known tsns
* avoid wrongly set startPosition
* move
* remove
* refactor
* typo
* fix
* fix changed path
2025-03-09 23:49:42 -07:00
chrislu
ec155022e7
"golang.org/x/exp/slices" => "slices" and go fmt
2024-12-19 19:25:06 -08:00
chrislu
f8a22038c5
passing along publisher name
2024-05-29 23:31:48 -07:00
chrislu
3622196881
go fmt
2024-05-20 11:05:18 -07:00
chrislu
5038577f7e
Merge branch 'master' into mq
2024-05-20 11:04:53 -07:00
chrislu
d218fe54fa
go fmt
2024-05-20 11:03:56 -07:00
chrislu
b1871427c3
persist consumer group offset
...
1. use one follower
2. read write consumer group offset
2024-05-19 00:46:12 -07:00
chrislu
4e3701df77
reset last error
2024-05-05 14:16:54 -07:00
chrislu
0f35b3a4ea
add example to publish a record
2024-05-02 08:48:51 -07:00
chrislu
68ade92d4f
publisher send schema
2024-04-12 23:36:15 -07:00
chrislu
5324e4288a
rename parameter
2024-04-12 22:33:00 -07:00
chrislu
66a878af39
add control message to data message
2024-03-31 16:35:46 -07:00
chrislu
546ae87c39
send is_close message to broker
2024-03-31 01:28:40 -07:00
chrislu
c9df613b6b
add publisher name for debugging
2024-03-31 00:19:16 -07:00
chrislu
da31e9b939
correctly wait for the publisher to finish
2024-03-21 09:08:53 -07:00
chrislu
7003ce7425
publisher is able to fully send data and receive acks
...
still needs to close the pipes cleanly
2024-03-20 12:25:40 -07:00
chrislu
33ab6cfcf7
adjust error
2024-03-07 10:44:09 -08:00
chrislu
ead27a4f59
publish init add followers
2024-02-29 09:39:12 -08:00
chrislu
1b4484bf0a
go fmt
2024-02-29 09:38:52 -08:00
Chris Lu
90fcde0e26
report EOF error
2024-02-05 17:47:34 -08:00
Chris Lu
07028a4864
minor
2024-02-05 17:30:20 -08:00
Chris Lu
4dc6681833
proxy requests to lock owner
2024-02-05 16:46:40 -08:00
chrislu
0b2e5ddc7c
wait 3 seconds before shutting down publish client, to wait for all messages to be received
2024-01-28 22:04:42 -08:00
chrislu
545d5fbdf6
unused code
2024-01-28 22:03:13 -08:00
chrislu
a297247137
wait for each publish clients
2024-01-28 14:27:32 -08:00
chrislu
b6c5e57c30
read will block if no items
2024-01-28 13:10:34 -08:00
chrislu
0bf5424a2e
adjust visibility
2024-01-28 13:09:30 -08:00
chrislu
dedfd31dfb
refactor
2024-01-28 12:30:08 -08:00
chrislu
cbf750a31f
refactor
2024-01-28 12:23:20 -08:00
chrislu
b51dfe2bff
wait for publishing clients
2024-01-28 12:06:58 -08:00
chrislu
f8c55f101e
clean up
2024-01-28 12:06:30 -08:00
chrislu
841fafd0a8
publish to input buffer
...
currently, the input buffer may not exist when start to publish
2024-01-27 23:43:22 -08:00
chrislu
91af1f3069
schedule jobs
2024-01-26 14:09:57 -08:00
chrislu
34a78ffad0
remove isForPublish from LookupTopicBrokers
...
also adds a return parameter: whether the topic exists or not
2024-01-16 08:52:42 -08:00
chrislu
026c54a9bb
fix publisher
2024-01-15 00:20:21 -08:00
chrislu
6a7a679137
passing partition generation timestamp
2024-01-10 22:36:17 -08:00
chrislu
d0d24f1e40
rename functions
2024-01-05 17:10:43 -08:00
chrislu
ee41dbb7fc
rename functions
2024-01-05 15:14:25 -08:00
Chris Lu
580940bf82
Merge accumulated changes related to message queue ( #5098 )
...
* balance partitions on brokers
* prepare topic partition first and then publish, move partition
* purge unused APIs
* clean up
* adjust logs
* add BalanceTopics() grpc API
* configure topic
* configure topic command
* refactor
* repair missing partitions
* sequence of operations to ensure ordering
* proto to close publishers and consumers
* rename file
* topic partition versioned by unixTimeNs
* create local topic partition
* close publishers
* randomize the client name
* wait until no publishers
* logs
* close stop publisher channel
* send last ack
* comments
* comment
* comments
* support list of brokers
* add cli options
* Update .gitignore
* logs
* return io.eof directly
* refactor
* optionally create topic
* refactoring
* detect consumer disconnection
* sub client wait for more messages
* subscribe by time stamp
* rename
* rename to sub_balancer
* rename
* adjust comments
* rename
* fix compilation
* rename
* rename
* SubscriberToSubCoordinator
* sticky rebalance
* go fmt
* add tests
* balance partitions on brokers
* prepare topic partition first and then publish, move partition
* purge unused APIs
* clean up
* adjust logs
* add BalanceTopics() grpc API
* configure topic
* configure topic command
* refactor
* repair missing partitions
* sequence of operations to ensure ordering
* proto to close publishers and consumers
* rename file
* topic partition versioned by unixTimeNs
* create local topic partition
* close publishers
* randomize the client name
* wait until no publishers
* logs
* close stop publisher channel
* send last ack
* comments
* comment
* comments
* support list of brokers
* add cli options
* Update .gitignore
* logs
* return io.eof directly
* refactor
* optionally create topic
* refactoring
* detect consumer disconnection
* sub client wait for more messages
* subscribe by time stamp
* rename
* rename to sub_balancer
* rename
* adjust comments
* rename
* fix compilation
* rename
* rename
* SubscriberToSubCoordinator
* sticky rebalance
* go fmt
* add tests
* tracking topic=>broker
* merge
* comment
2023-12-11 12:05:54 -08:00
chrislu
411bdda08d
fix compilation
2023-09-19 16:22:41 -07:00
chrislu
39941edc0b
add publisher shutdown
2023-09-07 23:55:19 -07:00
chrislu
984b6c54cf
ack interval 128
2023-09-06 23:15:29 -07:00
chrislu
9e4f985698
publish, benchmark
2023-09-04 21:43:30 -07:00
chrislu
cb470d44df
can pub and sub
2023-09-01 00:36:51 -07:00
chrislu
1eb2da46d5
connect and publish
2023-08-28 09:02:12 -07:00
chrislu
4d6c18d86f
pub sub initial tests
2023-08-27 17:50:59 -07:00
chrislu
1990456670
sub
2023-08-27 13:13:14 -07:00
