8
Ich habe ein Problem mit der Einführung von Sellerie Aufgaben. Wenn Sie den folgenden Befehl ein lokal celery -A codebase worker --loglevel=debug --beat laufen bekomme ich folgende AntwortWorker startet, schaltet sich dann aus. Keine Fehler
/local/lib/python2.7/site-packages/twisted/internet/endpoints.py:30: DeprecationWarning: twisted.internet.interfaces.IStreamClientEndpointStringParser was deprecated in Twisted 14.0.0: This interface has been superseded by IStreamClientEndpointStringParserWithReactor.
from twisted.internet.interfaces import (
DEBUG 2014-09-05 14:58:02,450 base 25908 139809439647552 Configuring Raven for host: http://69a23c7e80df4276822e771e1dec86e4:[email protected]:9000:/10
[2014-09-05 14:58:02,723: DEBUG/MainProcess] | Worker: Preparing bootsteps.
[2014-09-05 14:58:02,726: DEBUG/MainProcess] | Worker: Building graph...
[2014-09-05 14:58:02,727: DEBUG/MainProcess] | Worker: New boot order: {Beat, Timer, Hub, Queues (intra), Pool, Autoreloader, StateDB, Autoscaler, Consumer}
[2014-09-05 14:58:02,739: DEBUG/MainProcess] | Consumer: Preparing bootsteps.
[2014-09-05 14:58:02,740: DEBUG/MainProcess] | Consumer: Building graph...
[2014-09-05 14:58:02,757: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Agent, Events, Mingle, Gossip, Heart, Tasks, Control, event loop}
-------------- [email protected] v3.1.13 (Cipater)
---- **** -----
--- * *** * -- Linux-3.13.0-35-generic-x86_64-with-Ubuntu-14.04-trusty
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app: codebase:0x7f27e88a4f90
- ** ---------- .> transport: django://localhost//
- ** ---------- .> results: djcelery.backends.database:DatabaseBackend
- *** --- * --- .> concurrency: 2 (prefork)
-- ******* ----
--- ***** ----- [queues]
-------------- .> celery exchange=celery(direct) key=celery
[tasks]
. celery.backend_cleanup
. celery.chain
. celery.chord
. celery.chord_unlock
. celery.chunks
. celery.group
. celery.map
. celery.starmap
. geoloc.tasks.get_real_geolocation
. sofiatraffic_crawler.tasks.crawl_sofitraffic
. sofiatraffic_data.tasks.delete_stop
. sofiatraffic_data.tasks.delete_transport
[2014-09-05 14:58:02,767: DEBUG/MainProcess] | Worker: Starting Beat
[2014-09-05 14:58:02,771: DEBUG/MainProcess] ^-- substep ok
[2014-09-05 14:58:02,772: DEBUG/MainProcess] | Worker: Starting Pool
[2014-09-05 14:58:02,777: INFO/Beat] beat: Starting...
[2014-09-05 14:58:02,780: DEBUG/Beat] DatabaseScheduler: intial read
[2014-09-05 14:58:02,781: INFO/Beat] Writing entries...
[2014-09-05 14:58:02,805: DEBUG/MainProcess] ^-- substep ok
[2014-09-05 14:58:02,806: DEBUG/MainProcess] | Worker: Starting Consumer
[2014-09-05 14:58:02,806: DEBUG/MainProcess] | Consumer: Starting Connection
[2014-09-05 14:58:02,818: INFO/MainProcess] Connected to django://localhost//
[2014-09-05 14:58:02,819: DEBUG/MainProcess] ^-- substep ok
[2014-09-05 14:58:02,819: DEBUG/MainProcess] | Consumer: Starting Events
[2014-09-05 14:58:02,829: DEBUG/MainProcess] ^-- substep ok
[2014-09-05 14:58:02,829: DEBUG/Beat] DatabaseScheduler: Fetching database schedule
[2014-09-05 14:58:02,832: DEBUG/MainProcess] | Consumer: Starting Heart
[2014-09-05 14:58:02,832: DEBUG/MainProcess] ^-- substep ok
[2014-09-05 14:58:02,833: DEBUG/MainProcess] | Consumer: Starting Tasks
[2014-09-05 14:58:02,865: DEBUG/MainProcess] ^-- substep ok
[2014-09-05 14:58:02,866: DEBUG/MainProcess] | Consumer: Starting Control
[2014-09-05 14:58:02,871: DEBUG/Beat] Current schedule:
<ModelEntry: scrap-sofitraffic-every-day sofiatraffic_crawler.tasks.crawl_sofitraffic(*[], **{}) {4}>
<ModelEntry: delete-stop-every-day sofiatraffic_data.tasks.delete_stop(*[], **{}) {4}>
<ModelEntry: celery.backend_cleanup celery.backend_cleanup(*[], **{}) {4}>
<ModelEntry: delete-transport-every-day sofiatraffic_data.tasks.delete_transport(*[], **{}) {4}>
<ModelEntry: get-real-location-every-day geoloc.tasks.get_real_geolocation(*[], **{}) {4}>
[2014-09-05 14:58:02,873: DEBUG/MainProcess] ^-- substep ok
[2014-09-05 14:58:02,873: DEBUG/MainProcess] | Consumer: Starting event loop
[2014-09-05 14:58:02,874: WARNING/MainProcess] [email protected] ready.
[2014-09-05 14:58:03,540: INFO/Beat] DatabaseScheduler: Schedule changed.
[2014-09-05 14:58:03,541: INFO/Beat] Writing entries...
[2014-09-05 14:58:03,541: DEBUG/Beat] DatabaseScheduler: Fetching database schedule
[2014-09-05 14:58:03,557: DEBUG/Beat] Current schedule:
<ModelEntry: scrap-sofitraffic-every-day sofiatraffic_crawler.tasks.crawl_sofitraffic(*[], **{}) {4}>
<ModelEntry: delete-stop-every-day sofiatraffic_data.tasks.delete_stop(*[], **{}) {4}>
<ModelEntry: celery.backend_cleanup celery.backend_cleanup(*[], **{}) {4}>
<ModelEntry: delete-transport-every-day sofiatraffic_data.tasks.delete_transport(*[], **{}) {4}>
<ModelEntry: get-real-location-every-day geoloc.tasks.get_real_geolocation(*[], **{}) {4}>
[2014-09-05 14:58:03,558: DEBUG/Beat] beat: Ticking with max interval->5.00 seconds
[2014-09-05 14:58:03,573: DEBUG/Beat] beat: Waking up in 4.99 seconds.
Alles ist in Ordnung, der Prozess wird fortgesetzt und gehen. an den Server übertragen Dann wird mit dem gleichen Befehl ausführen, und ich bekomme die folgende Antwort:
/local/lib/python2.7/site-packages/twisted/internet/endpoints.py:30: DeprecationWarning: twisted.internet.interfaces.IStreamClientEndpointStringParser was deprecated in Twisted 14.0.0: This interface has been superseded by IStreamClientEndpointStringParserWithReactor.
from twisted.internet.interfaces import (
DEBUG 2014-09-05 15:03:14,511 base 7100 140161706530560 Configuring Raven for host: http://69a23c7e80df4276822e771e1dec86e4:[email protected]:9000:/10
[2014-09-05 15:03:14,694: DEBUG/MainProcess] | Worker: Preparing bootsteps.
[2014-09-05 15:03:14,696: DEBUG/MainProcess] | Worker: Building graph...
[2014-09-05 15:03:14,696: DEBUG/MainProcess] | Worker: New boot order: {Beat, Timer, Hub, Queues (intra), Pool, Autoreloader, Autoscaler, StateDB, Consumer}
[2014-09-05 15:03:14,723: DEBUG/MainProcess] | Consumer: Preparing bootsteps.
[2014-09-05 15:03:14,723: DEBUG/MainProcess] | Consumer: Building graph...
[2014-09-05 15:03:14,731: DEBUG/MainProcess] | Consumer: New boot order: {Connection, Events, Mingle, Tasks, Control, Gossip, Agent, Heart, event loop}
-------------- [email protected] v3.1.13 (Cipater)
---- **** -----
--- * *** * -- Linux-3.2.0-34-generic-x86_64-with-Ubuntu-12.04-precise
-- * - **** ---
- ** ---------- [config]
- ** ---------- .> app: codebase:0x138e5d0
- ** ---------- .> transport: django://localhost//
- ** ---------- .> results: djcelery.backends.database:DatabaseBackend
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ----
--- ***** ----- [queues]
-------------- .> celery exchange=celery(direct) key=celery
[tasks]
. celery.backend_cleanup
. celery.chain
. celery.chord
. celery.chord_unlock
. celery.chunks
. celery.group
. celery.map
. celery.starmap
[2014-09-05 15:03:14,735: DEBUG/MainProcess] | Worker: Starting Beat
[2014-09-05 15:03:14,737: DEBUG/MainProcess] ^-- substep ok
[2014-09-05 15:03:14,737: DEBUG/MainProcess] | Worker: Starting Pool
[2014-09-05 15:03:14,739: INFO/Beat] beat: Starting...
[2014-09-05 15:03:14,751: DEBUG/MainProcess] ^-- substep ok
[2014-09-05 15:03:14,751: DEBUG/MainProcess] | Worker: Starting Consumer
[2014-09-05 15:03:14,751: DEBUG/MainProcess] | Consumer: Starting Connection
[2014-09-05 15:03:14,756: INFO/MainProcess] Connected to django://localhost//
[2014-09-05 15:03:14,756: DEBUG/MainProcess] ^-- substep ok
[2014-09-05 15:03:14,756: DEBUG/MainProcess] | Consumer: Starting Events
[2014-09-05 15:03:14,758: DEBUG/Beat] DatabaseScheduler: intial read
[2014-09-05 15:03:14,758: INFO/Beat] Writing entries...
[2014-09-05 15:03:14,760: DEBUG/MainProcess] ^-- substep ok
[2014-09-05 15:03:14,760: DEBUG/MainProcess] | Consumer: Starting Tasks
[2014-09-05 15:03:14,763: DEBUG/Beat] DatabaseScheduler: Fetching database schedule
[2014-09-05 15:03:14,766: DEBUG/Beat] Current schedule:
[2014-09-05 15:03:14,772: DEBUG/MainProcess] | Worker: Closing Beat...
[2014-09-05 15:03:14,772: DEBUG/MainProcess] | Worker: Closing Pool...
[2014-09-05 15:03:15,046: DEBUG/Beat] beat: Ticking with max interval->5.00 seconds
[2014-09-05 15:03:15,052: DEBUG/Beat] beat: Waking up in 4.99 seconds.
[2014-09-05 15:03:15,549: DEBUG/MainProcess] | Worker: Closing Consumer...
[2014-09-05 15:03:15,549: DEBUG/MainProcess] | Worker: Stopping Consumer...
[2014-09-05 15:03:15,550: DEBUG/MainProcess] | Worker: Stopping Pool...
[2014-09-05 15:03:15,778: DEBUG/MainProcess] | Worker: Stopping Beat...
[2014-09-05 15:03:15,779: INFO/MainProcess] beat: Shutting down...
[2014-09-05 15:03:15,779: DEBUG/MainProcess] | Consumer: Shutdown Heart...
[2014-09-05 15:03:15,779: INFO/Beat] Writing entries...
[2014-09-05 15:03:15,779: DEBUG/MainProcess] | Consumer: Shutdown Control...
[2014-09-05 15:03:15,779: DEBUG/MainProcess] | Consumer: Shutdown Tasks...
[2014-09-05 15:03:15,780: INFO/Beat] Writing entries...
[2014-09-05 15:03:15,780: DEBUG/MainProcess] | Consumer: Shutdown Events...
[2014-09-05 15:03:15,780: DEBUG/MainProcess] | Consumer: Shutdown Connection...
Und sobald Sie Arbeiter sofort stoppt, ohne Fehler, ohne etwas zu starten. Irgendeine Idee, was könnte das Problem sein?
Haben Sie andere Arbeiter auf dem Server? Sie verwenden '--beat' beim Starten eines Arbeiters. Es sollte nur eine Instanz davon geben. – ChillarAnand
Es gibt 'gunicorn: worker',' nginx: worker', der sich um die Sentry kümmert. – Krasimir
Haben Sie unterschiedliche Konfiguration in lokalen Sys & Server? Sieht so aus, als ob der erste Arbeiter mehr Aufgaben bearbeitet als der zweite Arbeiter. – ChillarAnand