Go to file
git 3ea43cba66
add list queue
2024-06-10 21:11:02 +08:00
backends change gomod 2024-06-01 11:35:00 +08:00
brokers/etcd add list queue 2024-06-10 21:11:02 +08:00
examples change gomod 2024-06-01 11:35:00 +08:00
locks/etcd add backend 2024-05-25 01:13:08 +08:00
task init task 2024-06-01 14:25:48 +08:00
.gitignore init commit 2024-05-19 20:09:59 +08:00
LICENSE Initial commit 2024-05-19 20:00:34 +08:00
README.md finish handleDelayedTask 2024-06-08 07:55:03 +08:00
go.mod change gomod 2024-06-01 11:35:00 +08:00
go.sum add backend 2024-05-25 01:03:13 +08:00

README.md

machinery-plugins

This project is intended to provide multi broker, lock, backend for the RichardKnop/machinery library. About Machinery is an asynchronous task queue/job queue based on distributed message passing. This plugins library offers additional functionalities and tools to help you better utilize and customize Machinery.

Features

This library provides the following features:

  • Brokers: add etcd / mysql
  • Locks add etcd / mysql
  • Backends: add etcd / mysql

you can combine whatever you want

Quick Start

Install the library:

go get git.ifooth.com/common/machinery-plugins@latest

Import the library in your code and use it to create tasks:

import (
    "github.com/RichardKnop/machinery/v2"
    redisbackend "github.com/RichardKnop/machinery/v2/backends/redis"
    eagerlock "github.com/RichardKnop/machinery/v2/locks/eager"
    etcdbroker "git.ifooth.com/common/machinery-plugins/brokers/etcd"
)

func main() {
    // Create server instance
	// broker := redisbroker.NewGR(cnf, []string{"localhost:6379"}, 1)
	broker, err := etcdbroker.New(cnf, "http://127.0.0.1:2379")
	if err != nil {
		return nil, err
	}

	lock := eagerlock.New()
	backend := redisbackend.NewGR(cnf, []string{"localhost:6379"}, 3)
	server := machinery.NewServer(cnf, broker, backend, lock)
}

Contributing

We welcome contributions in any form! If you have any issues, suggestions, or wish to add new features, let us know by creating an issue or pull request.

License

machinery-plugins is based on the Apache 2.0 protocol. Please refer to LICENSE for details.

设计

push -> pending_tasks/{queue}/{task_uuid} handle -> 获取keys-only, 判断是否有consumerlist/watch(可用key列表) -> 原子操作 -> handler -> delete(每次全量key N可以对吧lock)

delay -> delay_tasks/r{xx}-taskuuid handlr -> 按key排序获取最新的一个(可以大于当前时间) -> 原子put/delete操作 -> done (性能没有问题, 多个服务会有冲突,最多N-1可以对比Lock)