Some of you may have seen “The ‘<name>’ task processor queue reached <number> scheduled tasks.” warning messages, wondered what they mean, and if there is anything you can do about them.
Task processors have been in Asterisk for a long time. Since Asterisk v12, they have become more important because the PJSIP, sorcery, and stasis modules heavily rely on task processors. As a result, the symptoms of overloading your Asterisk installation have changed. An early symptom of an overloaded system is when task processors start queueing an excessive number of tasks to process. If the overload conditions are not relieved then later symptoms can include high CPU utilization and/or high memory consumption due to extremely large queues.
You can see some task processor statistics by executing the CLI command “core show taskprocessors”. Starting in Asterisk v13.8.0 task processors created by PJSIP, sorcery, and stasis have meaningful names instead of an opaque UUID string. Meaningful task processor names help identify modules in Asterisk that are bottle necks on your system.
Task processor created by res_pjsip_registrar.so for incoming endpoint registrations. Asterisk v13.10.0 eliminated these task processors to continue using the pjsip/distributor task processor the REGISTER request came in on.
Task processor created by res_pjsip.so. Miscellaneous PJSIP tasks that don’t have any other association. These task processors are assigned tasks in a round robin sequence. Before Asterisk v13.10.0 these task processors also handled the initial PJSIP message distribution if the message was not associated with any other task processor.
Task processor created by res_pjsip.so. PJSIP distributes incoming messages to these task processors if the message is not associated with any other task processor. The task processor assignment is determined by hashing the SIP request’s Call-ID and From tag. These task processors are new since Asterisk v13.10.0.
Task processor created by res_pjsip_messaging.so to send all out-of-dialog MESSAGE requests for PJSIP.
Task processor created by res_pjsip_outbound_registration.so to handle the endpoint’s outbound registration messages.
Task processor created by res_pjsip_session.so to handle incoming and outgoing sessions and their associated dialog messages. Asterisk v13.10.0 renamed the task processors for outgoing sessions to pjsip/outsess/<endpoint>. The task processors for incoming sessions were eliminated to continue using the pjsip/distributor task processor the initial INVITE came in on.
Task processor created by res_pjsip_session.so to handle messages associated with the endpoint’s outgoing session (Outgoing channels). This includes any messages associated with the session’s dialog.
Task processor created by res_pjsip_pubsub.so to handle the endpoint’s outbound PUBLISH and SUBSCRIBE dialog messages. Asterisk v13.10.0 stopped using these task processors for incoming PUBLISH and SUBSCRIBE dialogs to continue using the pjsip/distributor task processor the original request came in on.
Task processor created by res_pjsip_refer.so to handle the endpoint’s outbound REFER requests.
Task processor created by res_pjsip_transport_websocket.so to process WebRTC HTTP requests for PJSIP.
Used to process sorcery observer callbacks. Sorcery task processors use a self managed thread pool.
Stasis topic subscription mailbox. The task processor has a dedicated thread to process the task queue.
Stasis topic subscription thread pool mailbox. The task processor uses an available thread from the stasis thread pool to process the task queue.
What can you do?
* Disable features you do not need. It is important that if you do not need features like Homer(HEP), CDR, CEL, or AMI that you do not enable them and for HEP you shouldn’t load those modules either.
* If the task processor name happens to be of the ‘subm:<topic>’ form where the topic is your stasis application name then that task processor is servicing the stasis message bus communication with your application. One thing you can try is to register multiple copies of your stasis application and randomly spread calls to the different copies of the application. e.g., my_stasis_app1, my_stasis_app2,…
There may be other things you could do but they likely depend more on your call patterns and the services your Asterisk installation is performing.
No Comments Yet
The Digium Blog
- No items