Interpreter¶
The Pfp interpreter is quite simple: it uses py010parser
to parse
the template into an abstract-syntax-tree, and then handles each of
the nodes in the tree appropriately.
The main method for handling nodes is the _handle_node
function.
The _handle_node
function performs basic housekeeping, logging,
decides if the user should be dropped into the interactive debugger,
and of course, handles the node itself.
If a methods are not implemented to handle a certain AST node, an
pfp.errors.UnsupportedASTNode
error will be raised. Implemented
methods to handle AST node types are found in the _node_switch
dict:
self._node_switch = {
AST.FileAST: self._handle_file_ast,
AST.Decl: self._handle_decl,
AST.TypeDecl: self._handle_type_decl,
AST.ByRefDecl: self._handle_byref_decl,
AST.Struct: self._handle_struct,
AST.Union: self._handle_union,
AST.StructRef: self._handle_struct_ref,
AST.IdentifierType: self._handle_identifier_type,
AST.Typedef: self._handle_typedef,
AST.Constant: self._handle_constant,
AST.BinaryOp: self._handle_binary_op,
AST.Assignment: self._handle_assignment,
AST.ID: self._handle_id,
AST.UnaryOp: self._handle_unary_op,
AST.FuncDef: self._handle_func_def,
AST.FuncCall: self._handle_func_call,
AST.FuncDecl: self._handle_func_decl,
AST.ParamList: self._handle_param_list,
AST.ExprList: self._handle_expr_list,
AST.Compound: self._handle_compound,
AST.Return: self._handle_return,
AST.ArrayDecl: self._handle_array_decl,
AST.InitList: self._handle_init_list,
AST.If: self._handle_if,
AST.For: self._handle_for,
AST.While: self._handle_while,
AST.DeclList: self._handle_decl_list,
AST.Break: self._handle_break,
AST.Continue: self._handle_continue,
AST.ArrayRef: self._handle_array_ref,
AST.Enum: self._handle_enum,
AST.Switch: self._handle_switch,
AST.Cast: self._handle_cast,
AST.Typename: self._handle_typename,
AST.EmptyStatement: self._handle_empty_statement,
StructDecls: self._handle_struct_decls,
UnionDecls: self._handle_union_decls,
}
Interpreter Reference Documentation¶
Python format parser
-
pfp.interp.
LazyField
(lookup_name, scope)[source]¶ Super non-standard stuff here. Dynamically changing the base class using the scope and the lazy name when the class is instantiated. This works as long as the original base class is not directly inheriting from object (which we’re not, since our original base class is fields.Field).
-
class
pfp.interp.
PfpInterp
(debug=False, parser=None, int3=True)[source]¶ -
classmethod
add_native
(name, func, ret, interp=None, send_interp=False)[source]¶ Add the native python function
func
into the pfp interpreter with the namename
and return valueret
so that it can be called from within a template script.Note
The
@native
decorator exists to simplify this.All native functions must have the signature
def func(params, ctxt, scope, stream, coord [,interp])
, optionally allowing an interpreter param ifsend_interp
isTrue
.Example:
The example below defines a function
Sum
using theadd_native
method.import pfp.fields from pfp.fields import PYVAL def native_sum(params, ctxt, scope, stream, coord): return PYVAL(params[0]) + PYVAL(params[1]) pfp.interp.PfpInterp.add_native("Sum", native_sum, pfp.fields.Int64)
Parameters: - name (basestring) – The name the function will be exposed as in the interpreter.
- func (function) – The native python function that will be referenced.
- ret (type(pfp.fields.Field)) – The field class that the return value should be cast to.
- interp (pfp.interp.PfpInterp) – The specific pfp interpreter the function should be defined in.
- send_interp (bool) – If true, the current pfp interpreter will be added as an argument to the function.
-
classmethod
add_predefine
(template)[source]¶ Add a template that should be run prior to running any other templates. This is useful for predefining types, etc.
Parameters: template (basestring) – The template text (unicode is also fine here)
-
get_bitfield_direction
()[source]¶ Return if the bitfield direction
Note
This should be applied AFTER taking into account endianness.
-
get_bitfield_padded
()[source]¶ Return if the bitfield input/output stream should be padded
Returns: True/False
-
get_curr_lines
()[source]¶ Return the current line number in the template, as well as the surrounding source lines
-
get_filename
()[source]¶ Return the filename of the data that is currently being parsed
Returns: The name of the data file being parsed.
-
get_types
()[source]¶ Return a types object that will contain all of the typedefd structs’ classes.
Returns: Types object Example:
Create a new PNG_CHUNK object from a PNG_CHUNK type that was defined in a template:
types = interp.get_types() chunk = types.PNG_CHUNK()
-
load_template
(template)[source]¶ Load a template and all required predefines into this interpreter. Future calls to
parse
will not require the template to be parsed.
-
parse
(stream, template=None, predefines=True, orig_filename=None, keep_successful=False, printf=True)[source]¶ Parse the data stream using the template (e.g. parse the 010 template and interpret the template using the stream as the data source).
Stream: The input data stream Template: The template to parse the stream with Keep_successful: Return whatever was successfully parsed before an error. _pfp__error
will contain the exception (if one was raised)Parameters: printf (bool) – If False
, printfs will be noops (default=``True``)Returns: Pfp Dom
-
set_bitfield_direction
(val)[source]¶ Set the bitfields to parse from left to right (1), the default (None), or right to left (-1)
-
classmethod
-
class
pfp.interp.
PfpTypes
(interp, scope)[source]¶ A class to hold all typedefd types in a template. Note that types are instantiated by having them parse a null-stream. This means that type creation will not work correctly for complicated structs that have internal control-flow
-
class
pfp.interp.
Scope
(logger, parent=None)[source]¶ A class to keep track of the current scope of the interpreter
-
add_local
(field_name, field)[source]¶ Add a local variable in the current scope
Field_name: The field’s name Field: The field Returns: None
-
add_refd_struct_or_union
(name, refd_name, interp, node)[source]¶ Add a lazily-looked up typedef struct or union
Name: name of the typedefd struct/union Node: the typedef node Interp: the 010 interpreter
-
add_type
(new_name, orig_names)[source]¶ Record the typedefd name for orig_names. Resolve orig_names to their core names and save those.
New_name: TODO Orig_names: TODO Returns: TODO
-
add_type_struct_or_union
(name, interp, node)[source]¶ Store the node with the name. When it is instantiated, the node itself will be handled.
Name: name of the typedefd struct/union Node: the union/struct node Interp: the 010 interpreter
-
add_var
(field_name, field, root=False)[source]¶ Add a var to the current scope (vars are fields that parse the input stream)
Field_name: TODO Field: TODO Returns: TODO
-
clone
()[source]¶ Return a new Scope object that has the curr_scope pinned at the current one :returns: A new scope object
-
get_id
(name, recurse=True)[source]¶ Get the first id matching
name
. Will either be a local or a var.Name: TODO Returns: TODO
-
get_local
(name, recurse=True)[source]¶ Get the local field (search for it) from the scope stack. An alias for
get_var
Name: The name of the local field
-
get_type
(name, recurse=True)[source]¶ Get the names for the typename (created by typedef)
Name: The typedef’d name to resolve Returns: An array of resolved names associated with the typedef’d name
-
get_var
(name, recurse=True)[source]¶ Return the first var of name
name
in the current scope stack (remember, vars are the ones that parse the input stream)Name: The name of the id Recurse: Whether parent scopes should also be searched (defaults to True) Returns: TODO
-
-
pfp.interp.
StructUnionTypeRef
(curr_scope, typedef_name, refd_name, interp, node)[source]¶ Create a typedef that resolves itself dynamically. This is needed in situations like:
struct MY_STRUCT { char magic[4]; unsigned int filesize; }; typedef struct MY_STRUCT ME; LittleEndian(); ME s;
The typedef
ME
is handled before theMY_STRUCT
declaration actually occurs. The typedef value forME
should not the empty struct that is resolved, but should be a dynamically-looked up struct definition when aME
instance is actually declared.
Python format parser
-
pfp.interp.
LazyField
(lookup_name, scope)[source] Super non-standard stuff here. Dynamically changing the base class using the scope and the lazy name when the class is instantiated. This works as long as the original base class is not directly inheriting from object (which we’re not, since our original base class is fields.Field).
-
class
pfp.interp.
PfpInterp
(debug=False, parser=None, int3=True)[source] -
classmethod
add_native
(name, func, ret, interp=None, send_interp=False)[source] Add the native python function
func
into the pfp interpreter with the namename
and return valueret
so that it can be called from within a template script.Note
The
@native
decorator exists to simplify this.All native functions must have the signature
def func(params, ctxt, scope, stream, coord [,interp])
, optionally allowing an interpreter param ifsend_interp
isTrue
.Example:
The example below defines a function
Sum
using theadd_native
method.import pfp.fields from pfp.fields import PYVAL def native_sum(params, ctxt, scope, stream, coord): return PYVAL(params[0]) + PYVAL(params[1]) pfp.interp.PfpInterp.add_native("Sum", native_sum, pfp.fields.Int64)
Parameters: - name (basestring) – The name the function will be exposed as in the interpreter.
- func (function) – The native python function that will be referenced.
- ret (type(pfp.fields.Field)) – The field class that the return value should be cast to.
- interp (pfp.interp.PfpInterp) – The specific pfp interpreter the function should be defined in.
- send_interp (bool) – If true, the current pfp interpreter will be added as an argument to the function.
-
classmethod
add_predefine
(template)[source] Add a template that should be run prior to running any other templates. This is useful for predefining types, etc.
Parameters: template (basestring) – The template text (unicode is also fine here)
-
cont
()[source] Continue the interpreter
-
classmethod
define_natives
()[source] Define the native functions for PFP
-
eval
(statement, ctxt=None)[source] Eval a single statement (something returnable)
-
get_bitfield_direction
()[source] Return if the bitfield direction
Note
This should be applied AFTER taking into account endianness.
-
get_bitfield_padded
()[source] Return if the bitfield input/output stream should be padded
Returns: True/False
-
get_curr_lines
()[source] Return the current line number in the template, as well as the surrounding source lines
-
get_filename
()[source] Return the filename of the data that is currently being parsed
Returns: The name of the data file being parsed.
-
get_types
()[source] Return a types object that will contain all of the typedefd structs’ classes.
Returns: Types object Example:
Create a new PNG_CHUNK object from a PNG_CHUNK type that was defined in a template:
types = interp.get_types() chunk = types.PNG_CHUNK()
-
load_template
(template)[source] Load a template and all required predefines into this interpreter. Future calls to
parse
will not require the template to be parsed.
-
parse
(stream, template=None, predefines=True, orig_filename=None, keep_successful=False, printf=True)[source] Parse the data stream using the template (e.g. parse the 010 template and interpret the template using the stream as the data source).
Stream: The input data stream Template: The template to parse the stream with Keep_successful: Return whatever was successfully parsed before an error. _pfp__error
will contain the exception (if one was raised)Parameters: printf (bool) – If False
, printfs will be noops (default=``True``)Returns: Pfp Dom
-
set_bitfield_direction
(val)[source] Set the bitfields to parse from left to right (1), the default (None), or right to left (-1)
-
set_bitfield_padded
(val)[source] Set if the bitfield input/output stream should be padded
Val: True/False Returns: None
-
set_break
(break_type)[source] Set if the interpreter should break.
Returns: TODO
-
step_into
()[source] Step over/into the next statement
-
step_over
()[source] Perform one step of the interpreter
-
classmethod
-
class
pfp.interp.
PfpTypes
(interp, scope)[source] A class to hold all typedefd types in a template. Note that types are instantiated by having them parse a null-stream. This means that type creation will not work correctly for complicated structs that have internal control-flow
-
class
pfp.interp.
Scope
(logger, parent=None)[source] A class to keep track of the current scope of the interpreter
-
add_local
(field_name, field)[source] Add a local variable in the current scope
Field_name: The field’s name Field: The field Returns: None
-
add_refd_struct_or_union
(name, refd_name, interp, node)[source] Add a lazily-looked up typedef struct or union
Name: name of the typedefd struct/union Node: the typedef node Interp: the 010 interpreter
-
add_type
(new_name, orig_names)[source] Record the typedefd name for orig_names. Resolve orig_names to their core names and save those.
New_name: TODO Orig_names: TODO Returns: TODO
-
add_type_class
(name, cls)[source] Store the class with the name
-
add_type_struct_or_union
(name, interp, node)[source] Store the node with the name. When it is instantiated, the node itself will be handled.
Name: name of the typedefd struct/union Node: the union/struct node Interp: the 010 interpreter
-
add_var
(field_name, field, root=False)[source] Add a var to the current scope (vars are fields that parse the input stream)
Field_name: TODO Field: TODO Returns: TODO
-
clear_meta
()[source] Clear metadata about the current statement
-
clone
()[source] Return a new Scope object that has the curr_scope pinned at the current one :returns: A new scope object
-
get_id
(name, recurse=True)[source] Get the first id matching
name
. Will either be a local or a var.Name: TODO Returns: TODO
-
get_local
(name, recurse=True)[source] Get the local field (search for it) from the scope stack. An alias for
get_var
Name: The name of the local field
-
get_meta
(meta_name)[source] Get the current meta value named
meta_name
-
get_type
(name, recurse=True)[source] Get the names for the typename (created by typedef)
Name: The typedef’d name to resolve Returns: An array of resolved names associated with the typedef’d name
-
get_var
(name, recurse=True)[source] Return the first var of name
name
in the current scope stack (remember, vars are the ones that parse the input stream)Name: The name of the id Recurse: Whether parent scopes should also be searched (defaults to True) Returns: TODO
-
level
()[source] Return the current scope level
-
pop
()[source] Leave the current scope :returns: TODO
-
pop_meta
(name)[source] Pop metadata about the current statement from the metadata stack for the current statement.
Name: The name of the metadata
-
push
(new_scope=None)[source] Create a new scope :returns: TODO
-
push_meta
(meta_name, meta_value)[source] Push metadata about the current statement onto the metadata stack for the current statement. Mostly used for tracking integer promotion and casting types
-
-
pfp.interp.
StructUnionTypeRef
(curr_scope, typedef_name, refd_name, interp, node)[source] Create a typedef that resolves itself dynamically. This is needed in situations like:
struct MY_STRUCT { char magic[4]; unsigned int filesize; }; typedef struct MY_STRUCT ME; LittleEndian(); ME s;
The typedef
ME
is handled before theMY_STRUCT
declaration actually occurs. The typedef value forME
should not the empty struct that is resolved, but should be a dynamically-looked up struct definition when aME
instance is actually declared.