plaso.multi_processing package¶
Submodules¶
plaso.multi_processing.analysis_process module¶
plaso.multi_processing.base_process module¶
Base class for a process used in multi-processing.
-
class
plaso.multi_processing.base_process.
MultiProcessBaseProcess
(processing_configuration, enable_sigsegv_handler=False, **kwargs)[source]¶ Bases:
multiprocessing.context.Process
Multi-processing process interface.
-
rpc_port
¶ port number of the process status RPC server.
- Type
int
-
name
¶ process name.
- Type
str
-
plaso.multi_processing.engine module¶
plaso.multi_processing.logger module¶
The multi-processing sub module logger.
plaso.multi_processing.plaso_xmlrpc module¶
XML RPC server and client.
-
class
plaso.multi_processing.plaso_xmlrpc.
ThreadedXMLRPCServer
(callback)[source]¶ Bases:
plaso.multi_processing.rpc.RPCServer
Threaded XML RPC server.
-
class
plaso.multi_processing.plaso_xmlrpc.
XMLProcessStatusRPCClient
[source]¶ Bases:
plaso.multi_processing.plaso_xmlrpc.XMLRPCClient
XML process status RPC client.
-
class
plaso.multi_processing.plaso_xmlrpc.
XMLProcessStatusRPCServer
(callback)[source]¶ Bases:
plaso.multi_processing.plaso_xmlrpc.ThreadedXMLRPCServer
XML process status threaded RPC server.
-
class
plaso.multi_processing.plaso_xmlrpc.
XMLRPCClient
[source]¶ Bases:
plaso.multi_processing.rpc.RPCClient
XML RPC client.
plaso.multi_processing.psort module¶
plaso.multi_processing.rpc module¶
The RPC client and server interface.
-
class
plaso.multi_processing.rpc.
RPCServer
(callback)[source]¶ Bases:
object
RPC server interface.
plaso.multi_processing.task_engine module¶
plaso.multi_processing.task_manager module¶
The task manager.
-
class
plaso.multi_processing.task_manager.
TaskManager
[source]¶ Bases:
object
Manages tasks and tracks their completion and status.
A task being tracked by the manager must be in exactly one of the following states:
- abandoned: a task assumed to be abandoned because a tasks that has been
queued or was processing exceeds the maximum inactive time.
merging: a task that is being merged by the engine.
- pending_merge: the task has been processed and is ready to be merged with
the session storage.
- processed: a worker has completed processing the task, but it is not ready
to be merged into the session storage.
processing: a worker is processing the task.
- queued: the task is waiting for a worker to start processing it. It is also
possible that a worker has already completed the task, but no status update was collected from the worker while it processed the task.
Once the engine reports that a task is completely merged, it is removed from the task manager.
Tasks are considered “pending” when there is more work that needs to be done to complete these tasks. Pending applies to tasks that are: * not abandoned; * abandoned, but need to be retried.
Abandoned tasks without corresponding retry tasks are considered “failed” when the foreman is done processing.
-
CheckTaskToMerge
(task)[source]¶ Checks if the task should be merged.
- Parameters
task (Task) – task.
- Returns
True if the task should be merged.
- Return type
bool
- Raises
KeyError – if the task was not queued, processing or abandoned.
-
CompleteTask
(task)[source]¶ Completes a task.
The task is complete and can be removed from the task manager.
- Parameters
task (Task) – task.
- Raises
KeyError – if the task was not merging.
-
CreateRetryTask
()[source]¶ Creates a task that to retry a previously abandoned task.
- Returns
- a task that was abandoned but should be retried or None if there are
no abandoned tasks that should be retried.
- Return type
-
CreateTask
(session_identifier)[source]¶ Creates a task.
- Parameters
session_identifier (str) – the identifier of the session the task is part of.
- Returns
task attribute container.
- Return type
-
GetFailedTasks
()[source]¶ Retrieves all failed tasks.
Failed tasks are tasks that were abandoned and have no retry task once the foreman is done processing.
- Returns
tasks.
- Return type
list[Task]
-
GetProcessedTaskByIdentifier
(task_identifier)[source]¶ Retrieves a task that has been processed.
- Parameters
task_identifier (str) – unique identifier of the task.
- Returns
a task that has been processed.
- Return type
- Raises
KeyError – if the task was not processing, queued or abandoned.
-
GetStatusInformation
()[source]¶ Retrieves status information about the tasks.
- Returns
tasks status information.
- Return type
-
GetTaskPendingMerge
(current_task)[source]¶ Retrieves the first task that is pending merge or has a higher priority.
This function will check if there is a task with a higher merge priority than the current_task being merged. If so, that task with the higher priority is returned.
-
HasPendingTasks
()[source]¶ Determines if there are tasks running or in need of retrying.
- Returns
- True if there are tasks that are active, ready to be merged or
need to be retried.
- Return type
bool
-
RemoveTask
(task)[source]¶ Removes an abandoned task.
- Parameters
task (Task) – task.
- Raises
KeyError – if the task was not abandoned or the task was abandoned and was not retried.
-
SampleTaskStatus
(task, status)[source]¶ Takes a sample of the status of the task for profiling.
- Parameters
task (Task) – a task.
status (str) – status.
-
StartProfiling
(configuration, identifier)[source]¶ Starts profiling.
- Parameters
configuration (ProfilingConfiguration) – profiling configuration.
identifier (str) – identifier of the profiling session used to create the sample filename.