i see common pattern services try develop , wonder if there tools / libraries out there here. while default jobs discussed in microservice literature request -> response nature, our jobs more or less assignments of semi permanent tasks.
examples of such tasks
- listen on message queue data source x , y, correlate data comes in , store in z.
- keep in-memory buffer calculates running average of past 15 mins of data everytime new data entry comes in.
currently our services written in php. due perceived overhead of php processes , connections message queue we'd single service process handle multiple of jobs simultanously.
a chart illustrated setup have in our head: 
- service workers deamonized php scripts
- for service registry looking @ zookeeper
while zookeeper (and curator) loadbalancing, did not find around distributing permanent jobs (that updatable, removable, , must reassigned when worker dies)
proposed responsibilities of job manager
- knows jobs
- knows services can these jobs
- can assign jobs services
- can send job updates services
- can reassign jobs if worker dies
are there libraries / tools can tackle such problems, , can function job manager? or 1 big anti pattern , should other way?
you should have @ gearman.
it composes of client assigns jobs, 1 or more workers pick , execute jobs , server maintain list of functions (services) , jobs pending. re-assign jobs if worker dies.
Comments
Post a Comment