plaso.storage package

Subpackages

• plaso.storage.fake package
  • Submodules
  • plaso.storage.fake.writer module
  • Module contents
• plaso.storage.sqlite package
  • Submodules
  • plaso.storage.sqlite.merge_reader module
  • plaso.storage.sqlite.reader module
  • plaso.storage.sqlite.sqlite_file module
  • plaso.storage.sqlite.writer module
  • Module contents

Submodules

plaso.storage.event_heaps module

Heaps to sort events in chronological order.

class plaso.storage.event_heaps.BaseEventHeap

Bases: object

Event heap interface.

PopEvent()

Pops an event from the heap.

Returns:event. Return type:EventObject

PopEvents()

Pops events from the heap.

Yields:EventObject – event.

PushEvent(event)

Pushes an event onto the heap.

Parameters:event (EventObject) – event.

PushEvents(events)

Pushes events onto the heap.

Parameters:list[EventObject] (events) – events.

number_of_events

int – number of serialized events on the heap.

class plaso.storage.event_heaps.EventHeap

Bases: plaso.storage.event_heaps.BaseEventHeap

Event heap.

PopEvent()

Pops an event from the heap.

Returns:event. Return type:EventObject

PushEvent(event)

Pushes an event onto the heap.

Parameters:event (EventObject) – event.

class plaso.storage.event_heaps.SerializedEventHeap

Bases: object

Serialized event heap.

data_size

int – total data size of the serialized events on the heap.

Empty()

Empties the heap.

PopEvent()

Pops an event from the heap.

Returns:containing:

int: event timestamp or None if the heap is empty bytes: serialized event or None if the heap is empty

Return type:tuple

PushEvent(timestamp, event_data)

Pushes a serialized event onto the heap.

Parameters:

• timestamp (int) – event timestamp, which contains the number of micro seconds since January 1, 1970, 00:00:00 UTC.
• event_data (bytes) – serialized event.

number_of_events

int – number of serialized events on the heap.

plaso.storage.event_tag_index module

The event tag index.

class plaso.storage.event_tag_index.EventTagIndex

Bases: object

Event tag index.

The event tag index is used to map event tags to events.

It is necessary for the ZIP storage files since previously stored event tags cannot be altered.

GetEventTagByIdentifier(storage_file, event_identifier)

Retrieves the most recently updated event tag for an event.

Parameters:

• storage_file (BaseStorageFile) – storage file.
• event_identifier (AttributeContainerIdentifier) – event attribute container identifier.

Returns:

event tag or None if the event has no event tag.

Return type:

EventTag

SetEventTag(event_tag)

Sets an event tag in the index.

Parameters:event_tag (EventTag) – event tag.

plaso.storage.factory module

This file contains the storage factory class.

class plaso.storage.factory.StorageFactory

Bases: object

Storage factory.

classmethod CreateStorageFile(storage_format)

Creates a storage file.

Parameters:storage_format (str) – storage format. Returns:

a storage file or None if the storage file cannot be

opened or the storage format is not supported.

Return type:StorageFile

classmethod CreateStorageFileForFile(path)

Creates a storage file based on the file.

Parameters:path (str) – path to the storage file. Returns:

a storage file or None if the storage file cannot be

opened or the storage format is not supported.

Return type:StorageFile

classmethod CreateStorageReaderForFile(path)

Creates a storage reader based on the file.

Parameters:path (str) – path to the storage file. Returns:

a storage reader or None if the storage file cannot be

opened or the storage format is not supported.

Return type:StorageReader

classmethod CreateStorageWriter(storage_format, session, path)

Creates a storage writer.

Parameters:

• session (Session) – session the storage changes are part of.
• path (str) – path to the storage file.
• storage_format (str) – storage format.

Returns:

a storage writer or None if the storage file cannot be

opened or the storage format is not supported.

Return type:

StorageWriter

classmethod CreateStorageWriterForFile(session, path)

Creates a storage writer based on the file.

Parameters:

• session (Session) – session the storage changes are part of.
• path (str) – path to the storage file.

Returns:

a storage writer or None if the storage file cannot be

opened or the storage format is not supported.

Return type:

StorageWriter

plaso.storage.identifiers module

Storage attribute container identifier objects.

class plaso.storage.identifiers.FakeIdentifier(attribute_values_hash)

Bases: plaso.containers.interface.AttributeContainerIdentifier

Fake attribute container identifier intended for testing.

attribute_values_hash

int – hash value of the attribute values.

CopyToString()

Copies the identifier to a string representation.

Returns:unique identifier or None. Return type:str

class plaso.storage.identifiers.SQLTableIdentifier(name, row_identifier)

Bases: plaso.containers.interface.AttributeContainerIdentifier

SQL table attribute container identifier.

The identifier is used to uniquely identify attribute containers. Where for example an attribute container is stored as a JSON serialized data in a SQLite database file.

name

str – name of the table.

row_identifier

int – unique identifier of the row in the table.

CopyToString()

Copies the identifier to a string representation.

Returns:unique identifier or None. Return type:str

class plaso.storage.identifiers.SerializedStreamIdentifier(stream_number, entry_index)

Bases: plaso.containers.interface.AttributeContainerIdentifier

Serialized stream attribute container identifier.

The identifier is used to uniquely identify attribute containers. Where for example an attribute container is stored as a JSON serialized data in a ZIP file.

stream_number

int – number of the serialized attribute container stream.

entry_index

int – number of the serialized event within the stream.

CopyToString()

Copies the identifier to a string representation.

Returns:unique identifier or None. Return type:str

plaso.storage.interface module

The storage interface classes.

class plaso.storage.interface.BaseStorageFile

Bases: plaso.storage.interface.BaseStore

Interface for file-based stores.

class plaso.storage.interface.BaseStore

Bases: object

Storage interface.

AddAnalysisReport(analysis_report)

Adds an analysis report.

Parameters:analysis_report (AnalysisReport) – analysis report.

AddError(error)

Adds an error.

Parameters:error (ExtractionError) – error.

AddEvent(event)

Adds an event.

Parameters:event (EventObject) – event.

AddEventSource(event_source)

Adds an event source.

Parameters:event_source (EventSource) – event source.

AddEventTag(event_tag)

Adds an event tag.

Parameters:event_tag (EventTag) – event tag.

Close()

Closes the storage.

GetAnalysisReports()

Retrieves the analysis reports.

Yields:AnalysisReport – analysis report.

GetErrors()

Retrieves the errors.

Yields:ExtractionError – error.

GetEventData()

Retrieves the event data.

Yields:EventData – event data.

GetEventDataByIdentifier(identifier)

Retrieves specific event data.

Parameters:identifier (AttributeContainerIdentifier) – event data identifier. Returns:event data or None if not available. Return type:EventData

GetEventSources()

Retrieves the event sources.

Yields:EventSource – event source.

GetEventTagByIdentifier(identifier)

Retrieves a specific event tag.

Parameters:identifier (AttributeContainerIdentifier) – event tag identifier. Returns:event tag or None if not available. Return type:EventTag

GetEventTags()

Retrieves the event tags.

Yields:EventTag – event tag.

GetEvents()

Retrieves the events.

Yields:EventObject – event.

GetNumberOfEventSources()

Retrieves the number event sources.

Returns:number of event sources. Return type:int

GetSortedEvents(time_range=None)

Retrieves the events in increasing chronological order.

This includes all events written to the storage including those pending being flushed (written) to the storage.

Parameters:time_range (Optional[TimeRange]) – time range used to filter events that fall in a specific period. Yields:EventObject – event.

HasAnalysisReports()

Determines if a store contains analysis reports.

Returns:True if the store contains analysis reports. Return type:bool

HasErrors()

Determines if a store contains extraction errors.

Returns:True if the store contains extraction errors. Return type:bool

HasEventTags()

Determines if a store contains event tags.

Returns:True if the store contains event tags. Return type:bool

Open(**kwargs)

Opens the storage.

ReadPreprocessingInformation(knowledge_base)

Reads preprocessing information.

The preprocessing information contains the system configuration which contains information about various system specific configuration data, for example the user accounts.

Parameters:knowledge_base (KnowledgeBase) – is used to store the preprocessing information.

SetSerializersProfiler(serializers_profiler)

Sets the serializers profiler.

Parameters:serializers_profiler (SerializersProfiler) – serializers profiler.

SetStorageProfiler(storage_profiler)

Sets the storage profiler.

Parameters:storage_profiler (StorageProfiler) – storage profiler.

WritePreprocessingInformation(knowledge_base)

Writes preprocessing information.

Parameters:knowledge_base (KnowledgeBase) – contains the preprocessing information.

WriteSessionCompletion(session_completion)

Writes session completion information.

Parameters:session_completion (SessionCompletion) – session completion information.

WriteSessionStart(session_start)

Writes session start information.

Parameters:session_start (SessionStart) – session start information.

WriteTaskCompletion(task_completion)

Writes task completion information.

Parameters:task_completion (TaskCompletion) – task completion information.

WriteTaskStart(task_start)

Writes task start information.

Parameters:task_start (TaskStart) – task start information.

class plaso.storage.interface.SerializedAttributeContainerList

Bases: object

Serialized attribute container list.

The list is unsorted and pops attribute containers in the same order as pushed to preserve order.

The GetAttributeContainerByIndex method should be used to read attribute containers from the list while it being filled.

data_size

int – total data size of the serialized attribute containers on the list.

next_sequence_number

int – next attribute container sequence number.

Empty()

Empties the list.

GetAttributeContainerByIndex(index)

Retrieves a specific serialized attribute container from the list.

Parameters:index (int) – attribute container index. Returns:serialized attribute container data or None if not available. Return type:bytes Raises:IndexError – if the index is less than zero.

PopAttributeContainer()

Pops a serialized attribute container from the list.

Returns:serialized attribute container data. Return type:bytes

PushAttributeContainer(serialized_data)

Pushes a serialized attribute container onto the list.

Parameters:serialized_data (bytes) – serialized attribute container data.

number_of_attribute_containers

int – number of serialized attribute containers on the list.

class plaso.storage.interface.StorageFileMergeReader(storage_writer)

Bases: plaso.storage.interface.StorageMergeReader

Storage reader interface for merging file-based stores.

class plaso.storage.interface.StorageFileReader(path)

Bases: plaso.storage.interface.StorageReader

File-based storage reader interface.

Close()

Closes the storage reader.

GetAnalysisReports()

Retrieves the analysis reports.

Returns:analysis report generator. Return type:generator(AnalysisReport)

GetErrors()

Retrieves the errors.

Returns:error generator. Return type:generator(ExtractionError)

GetEventData()

Retrieves the event data.

Returns:event data generator. Return type:generator(EventData)

GetEventDataByIdentifier(identifier)

Retrieves specific event data.

Parameters:identifier (AttributeContainerIdentifier) – event data identifier. Returns:event data or None if not available. Return type:EventData

GetEventSources()

Retrieves the event sources.

Returns:event source generator. Return type:generator(EventSource)

GetEventTagByIdentifier(identifier)

Retrieves a specific event tag.

Parameters:identifier (AttributeContainerIdentifier) – event tag identifier. Returns:event tag or None if not available. Return type:EventTag

GetEventTags()

Retrieves the event tags.

Returns:event tag generator. Return type:generator(EventTag)

GetEvents()

Retrieves the events.

Returns:event generator. Return type:generator(EventObject)

GetNumberOfAnalysisReports()

Retrieves the number analysis reports.

Returns:number of analysis reports. Return type:int

GetSortedEvents(time_range=None)

Retrieves the events in increasing chronological order.

This includes all events written to the storage including those pending being flushed (written) to the storage.

Parameters:time_range (Optional[TimeRange]) – time range used to filter events that fall in a specific period. Returns:event generator. Return type:generator(EventObject)

ReadPreprocessingInformation(knowledge_base)

Reads preprocessing information.

The preprocessing information contains the system configuration which contains information about various system specific configuration data, for example the user accounts.

Parameters:knowledge_base (KnowledgeBase) – is used to store the preprocessing information.

SetSerializersProfiler(serializers_profiler)

Sets the serializers profiler.

Parameters:serializers_profiler (SerializersProfiler) – serializers profiler.

SetStorageProfiler(storage_profiler)

Sets the storage profiler.

Parameters:storage_profiler (StorageProfiler) – storage profiler.

class plaso.storage.interface.StorageFileWriter(session, output_file, storage_type='session', task=None)

Bases: plaso.storage.interface.StorageWriter

Defines an interface for a file-backed storage writer.

AddAnalysisReport(analysis_report)

Adds an analysis report.

Parameters:analysis_report (AnalysisReport) – analysis report. Raises:IOError – when the storage writer is closed.

AddError(error)

Adds an error.

Parameters:error (AnalysisError|ExtractionError) – an analysis or extraction error. Raises:IOError – when the storage writer is closed.

AddEvent(event)

Adds an event.

Parameters:event (EventObject) – an event. Raises:IOError – when the storage writer is closed.

AddEventData(event_data)

Adds event data.

Parameters:event_data (EventData) – event data. Raises:IOError – when the storage writer is closed.

AddEventSource(event_source)

Adds an event source.

Parameters:event_source (EventSource) – an event source. Raises:IOError – when the storage writer is closed.

AddEventTag(event_tag)

Adds an event tag.

Parameters:event_tag (EventTag) – an event tag. Raises:IOError – when the storage writer is closed.

CheckTaskReadyForMerge(task)

Checks if a task is ready for merging with this session storage.

If the task is ready to be merged, this method also sets the task’s storage file size.

Parameters:task (Task) – task. Returns:True if the task is ready to be merged. Return type:bool Raises:IOError – if the storage type is not supported or if the temporary path for the task storage does not exist.

Close()

Closes the storage writer.

Raises:IOError – when the storage writer is closed.

CreateTaskStorage(task)

Creates a task storage.

The task storage is used to store attributes created by the task.

Parameters:task (Task) – task. Returns:storage writer. Return type:StorageWriter Raises:IOError – if the storage type is not supported.

FinalizeTaskStorage(task)

Finalizes a processed task storage.

Moves the task storage file from its temporary directory to the processed directory.

Parameters:task (Task) – task. Raises:IOError – if the storage type is not supported or if the storage file cannot be renamed.

GetEventDataByIdentifier(identifier)

Retrieves specific event data.

Parameters:identifier (AttributeContainerIdentifier) – event data identifier. Returns:event data or None if not available. Return type:EventData

GetEventTagByIdentifier(identifier)

Retrieves a specific event tag.

Parameters:identifier (AttributeContainerIdentifier) – event tag identifier. Returns:event tag or None if not available. Return type:EventTag

GetEventTags()

Retrieves the event tags.

Returns:event tag generator. Return type:generator(EventTag)

GetEvents()

Retrieves the events.

Returns:event generator. Return type:generator(EventObject) Raises:IOError – when the storage writer is closed.

GetFirstWrittenEventSource()

Retrieves the first event source that was written after open.

Using GetFirstWrittenEventSource and GetNextWrittenEventSource newly added event sources can be retrieved in order of addition.

Returns:event source or None if there are no newly written ones. Return type:EventSource Raises:IOError – when the storage writer is closed.

GetNextWrittenEventSource()

Retrieves the next event source that was written after open.

Returns:event source or None if there are no newly written ones. Return type:EventSource Raises:IOError – when the storage writer is closed.

GetProcessedTaskIdentifiers()

Identifiers for tasks which have been processed.

Returns:task identifiers that are processed. Return type:list[str] Raises:IOError – if the storage type is not supported or if the temporary path for the task storage does not exist.

GetSortedEvents(time_range=None)

Retrieves the events in increasing chronological order.

This includes all events written to the storage including those pending being flushed (written) to the storage.

Parameters:time_range (Optional[TimeRange]) – time range used to filter events that fall in a specific period. Returns:event generator. Return type:generator(EventObject) Raises:IOError – when the storage writer is closed.

Open()

Opens the storage writer.

Raises:IOError – if the storage writer is already opened.

PrepareMergeTaskStorage(task)

Prepares a task storage for merging.

Moves the task storage file from the processed directory to the merge directory.

Parameters:task (Task) – task. Raises:IOError – if the storage type is not supported or if the storage file cannot be renamed.

ReadPreprocessingInformation(knowledge_base)

Reads preprocessing information.

The preprocessing information contains the system configuration which contains information about various system specific configuration data, for example the user accounts.

Parameters:knowledge_base (KnowledgeBase) – is used to store the preprocessing information. Raises:IOError – when the storage writer is closed.

RemoveProcessedTaskStorage(task)

Removes a processed task storage.

Parameters:task (Task) – task. Raises:IOError – if the storage type is not supported or if the storage file cannot be removed.

SetSerializersProfiler(serializers_profiler)

Sets the serializers profiler.

Parameters:serializers_profiler (SerializersProfiler) – serializers profiler.

SetStorageProfiler(storage_profiler)

Sets the storage profiler.

Parameters:storage_profiler (StorageProfiler) – storage profiler.

StartMergeTaskStorage(task)

Starts a merge of a task storage with the session storage.

Parameters:task (Task) – task. Returns:storage merge reader of the task storage. Return type:StorageMergeReader Raises:IOError – if the storage file cannot be opened or if the storage type is not supported or if the temporary path for the task storage does not exist or if the temporary path for the task storage doe not refers to a file.

StartTaskStorage()

Creates a temporary path for the task storage.

Raises:IOError – if the storage type is not supported or if the temporary path for the task storage already exists.

StopTaskStorage(abort=False)

Removes the temporary path for the task storage.

The results of tasks will be lost on abort.

Parameters:abort (bool) – True to indicate the stop is issued on abort. Raises:IOError – if the storage type is not supported.

WritePreprocessingInformation(knowledge_base)

Writes preprocessing information.

Parameters:knowledge_base (KnowledgeBase) – contains the preprocessing information. Raises:IOError – if the storage type does not support writing preprocessing information or when the storage writer is closed.

WriteSessionCompletion(aborted=False)

Writes session completion information.

Parameters:aborted (Optional[bool]) – True if the session was aborted. Raises:IOError – if the storage type is not supported or when the storage writer is closed.

WriteSessionStart()

Writes session start information.

Raises:IOError – if the storage type is not supported or when the storage writer is closed.

WriteTaskCompletion(aborted=False)

Writes task completion information.

Parameters:aborted (Optional[bool]) – True if the session was aborted. Raises:IOError – if the storage type is not supported or when the storage writer is closed.

WriteTaskStart()

Writes task start information.

Raises:IOError – if the storage type is not supported or when the storage writer is closed.

class plaso.storage.interface.StorageMergeReader(storage_writer)

Bases: object

Storage reader interface for merging.

MergeAttributeContainers(callback=None, maximum_number_of_containers=0)

Reads attribute containers from a task storage file into the writer.

Parameters:

• callback (function[StorageWriter, AttributeContainer]) – function to call after each attribute container is deserialized.
• maximum_number_of_containers (Optional[int]) – maximum number of containers to merge, where 0 represent no limit.

Returns:

True if the entire task storage file has been merged.

Return type:

bool

class plaso.storage.interface.StorageReader

Bases: object

Storage reader interface.

Close()

Closes the storage reader.

GetAnalysisReports()

Retrieves the analysis reports.

Yields:AnalysisReport – analysis report.

GetErrors()

Retrieves the errors.

Yields:ExtractionError – error.

GetEventData()

Retrieves the event data.

Yields:EventData – event data.

GetEventDataByIdentifier(identifier)

Retrieves specific event data.

Parameters:identifier (AttributeContainerIdentifier) – event data identifier. Returns:event data or None if not available. Return type:EventData

GetEventSources()

Retrieves event sources.

Yields:EventSourceObject – event source.

GetEventTagByIdentifier(identifier)

Retrieves a specific event tag.

Parameters:identifier (AttributeContainerIdentifier) – event tag identifier. Returns:event tag or None if not available. Return type:EventTag

GetEventTags()

Retrieves the event tags.

Yields:EventTag – event tag.

GetEvents()

Retrieves the events.

Yields:EventObject – event.

GetNumberOfAnalysisReports()

Retrieves the number analysis reports.

Returns:number of analysis reports. Return type:int

GetSortedEvents(time_range=None)

Retrieves the events in increasing chronological order.

This includes all events written to the storage including those pending being flushed (written) to the storage.

Parameters:time_range (Optional[TimeRange]) – time range used to filter events that fall in a specific period. Yields:EventObject – event.

ReadPreprocessingInformation(knowledge_base)

Reads preprocessing information.

The preprocessing information contains the system configuration which contains information about various system specific configuration data, for example the user accounts.

Parameters:knowledge_base (KnowledgeBase) – is used to store the preprocessing information.

SetSerializersProfiler(serializers_profiler)

Sets the serializers profiler.

Parameters:serializers_profiler (SerializersProfiler) – serializers profiler.

SetStorageProfiler(storage_profiler)

Sets the storage profiler.

Parameters:storage_profiler (StorageProfiler) – storage profile.

__enter__()

Make usable with “with” statement.

__exit__(exception_type, value, traceback)

Make usable with “with” statement.

class plaso.storage.interface.StorageWriter(session, storage_type='session', task=None)

Bases: object

Storage writer interface.

number_of_analysis_reports

int – number of analysis reports written.

number_of_errors

int – number of errors written.

number_of_event_sources

int – number of event sources written.

number_of_event_tags

int – number of event tags written.

number_of_events

int – number of events written.

AddAnalysisReport(analysis_report)

Adds an analysis report.

Parameters:analysis_report (AnalysisReport) – a report.

AddError(error)

Adds an error.

Parameters:error (ExtractionError) – an error.

AddEvent(event)

Adds an event.

Parameters:event (EventObject) – an event.

AddEventSource(event_source)

Adds an event source.

Parameters:event_source (EventSource) – an event source.

AddEventTag(event_tag)

Adds an event tag.

Parameters:event_tag (EventTag) – an event tag.

Close()

Closes the storage writer.

CreateTaskStorage(task)

Creates a task storage.

Parameters:task (Task) – task. Returns:storage writer. Return type:StorageWriter Raises:NotImplementedError – since there is no implementation.

FinalizeTaskStorage(task)

Finalizes a processed task storage.

Parameters:task (Task) – task. Raises:NotImplementedError – since there is no implementation.

GetEventDataByIdentifier(identifier)

Retrieves specific event data.

Parameters:identifier (AttributeContainerIdentifier) – event data identifier. Returns:event data or None if not available. Return type:EventData

GetEvents()

Retrieves the events.

Yields:EventObject – event.

GetFirstWrittenEventSource()

Retrieves the first event source that was written after open.

Using GetFirstWrittenEventSource and GetNextWrittenEventSource newly added event sources can be retrieved in order of addition.

Returns:event source or None if there are no newly written ones. Return type:EventSource

GetNextWrittenEventSource()

Retrieves the next event source that was written after open.

Returns:event source or None if there are no newly written ones. Return type:EventSource

GetSortedEvents(time_range=None)

Retrieves the events in increasing chronological order.

This includes all events written to the storage including those pending being flushed (written) to the storage.

Parameters:time_range (Optional[TimeRange]) – time range used to filter events that fall in a specific period. Yields:EventObject – event.

Open()

Opens the storage writer.

PrepareMergeTaskStorage(task)

Prepares a task storage for merging.

Parameters:task (Task) – task. Raises:NotImplementedError – since there is no implementation.

ReadPreprocessingInformation(knowledge_base)

Reads preprocessing information.

The preprocessing information contains the system configuration which contains information about various system specific configuration data, for example the user accounts.

Parameters:knowledge_base (KnowledgeBase) – is used to store the preprocessing information.

RemoveProcessedTaskStorage(task)

Removes a processed task storage.

Parameters:task (Task) – task. Raises:NotImplementedError – since there is no implementation.

SetSerializersProfiler(serializers_profiler)

Sets the serializers profiler.

Parameters:serializers_profiler (SerializersProfiler) – serializers profiler.

SetStorageProfiler(storage_profiler)

Sets the storage profiler.

Parameters:storage_profiler (StorageProfiler) – storage profiler.

WritePreprocessingInformation(knowledge_base)

Writes preprocessing information.

Parameters:knowledge_base (KnowledgeBase) – contains the preprocessing information.

WriteSessionCompletion(aborted=False)

Writes session completion information.

Parameters:aborted (Optional[bool]) – True if the session was aborted.

WriteSessionStart()

Writes session start information.

WriteTaskCompletion(aborted=False)

Writes task completion information.

Parameters:aborted (Optional[bool]) – True if the session was aborted.

WriteTaskStart()

Writes task start information.

plaso.storage.logger module

The storage sub module logger.

plaso.storage.time_range module

Storage time range objects.

class plaso.storage.time_range.TimeRange(start_timestamp, end_timestamp)

Bases: object

Date and time range.

The timestamp are integers containing the number of microseconds since January 1, 1970, 00:00:00 UTC.

duration

int – duration of the range in microseconds.

end_timestamp

int – timestamp that marks the end of the range.

start_timestamp

int – timestamp that marks the start of the range.

Module contents