Uploader Expectations and Application Flows¶
This section describes how an end-user of Pacifica Python Uploader is expected to interact with the modules, classes and methods above, and, by extension, Pacifica Ingest and Pacifica Policy servers.
Keywords for the API
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD",
"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be
interpreted as described in [RFC 2119](https://www.ietf.org/rfc/rfc2119.txt).
Uploader Program Flow¶
- The uploader program MUST construct a new instance of the
pacifica.uploader.metadata.MetaUpdate
class. The new instance of thepacifica.uploader.metadata.MetaUpdate
class MAY be associated with zero or more of instances of thepacifica.uploader.metadata.MetaObj
class. Thepacifica.uploader.metadata.MetaObj.value
field MAY beNone
. The new instance of thepacifica.uploader.metadata.MetaUpdate
class MUST NOT be associated with any instances of thepacifica.uploader.metadata.FileObj
class. - To determine completeness, the new instance of the
pacifica.uploader.metadata.MetaUpdate
class SHOULD be validated using thepacifica.uploader.metadata.MetaData.is_valid()
method (inherited by thepacifica.uploader.metadata.MetaUpdate
sub-class). Then, the uploader program MUST call thepacifica.uploader.metadata.PolicyQuery.PolicyQuery.valid_metadata()
method. The new instance of thepacifica.uploader.metadata.MetaUpdate
class MUST be valid prior to bundling. - The uploader program MUST dereference the
pacifica.uploader.metadata.MetaObj.displayType
field to determine the mode of selection for thepacifica.uploader.metadata.MetaObj.value
field. The value of thepacifica.uploader.metadata.MetaObj.displayType
field is uploader-program-specific, i.e., the value MUST be defined by the uploader program. - The uploader program MUST assign a non-
None
value to eachpacifica.uploader.metadata.MetaData.query_results
field by calling thepacifica.uploader.metadata.MetaUpdate.query_results()
method. Thepacifica.uploader.metadata.MetaData.query_results
field is alist
. - The value of the
pacifica.uploader.metadata.MetaData.query_results
field MUST be rendered according to the uploader-program-specific definition that is interpreted from the value of thepacifica.uploader.metadata.MetaObj.displayFormat
field, e.g., in the Python programming language, by calling thestr.format
method or by leveraging a template engine, such as Cheetah or Jinja2. - The uploader program MAY call the
pacifica.uploader.metadata.MetaUpdate.query_results()
method for instances of thepacifica.uploader.metadata.MetaObj
class whosevalue
field is non-None
. - The uploader program MUST handle all instances
pacifica.uploader.metadata.MetaUpdate
class, regardless of validity, i.e., the uploader program MUST NOT reject an instance of thepacifica.uploader.metadata.MetaUpdate
class under any circumstances, e.g., if there are unsatisfied dependencies between instances of thepacifica.uploader.metadata.MetaData
class. - When the uploader program is ready for a given
pacifica.uploader.metadata.MetaObj.value
field to be selected, the uploader program MUST assign to thepacifica.uploader.metadata.MetaObj.value
field the value of thepacifica.uploader.metadata.MetaObj.valueField
field, and then call thepacifica.uploader.metadata.MetaObj.update_parents()
method. The effect of this operation is to update thepacifica.uploader.metadata.MetaObj.value
fields of associated and dependent instances of thepacifica.uploader.metadata.MetaObj
class. After modification, the new state of the instance of thepacifica.uploader.metadata.MetaUpdate.MetaUpdate
class SHOULD be displayed to the end-user, as previously discussed. - The uploader program MUST verify that
pacifica.uploader.metadata.MetaUpdate.MetaUpdate.is_valid() == True
. If the instance of thepacifica.uploader.metadata.MetaUpdate.MetaUpdate
class is not valid, then the uploader program MUST repeat the instructions in the paragraph 8. - The uploader program MUST call the
pacifica.uploader.metadata.PolicyQuery.PolicyQueryData.valid_metadata()
method to validate the instance of thepacifica.uploader.metadata.MetaUpdate.MetaUpdate
class prior to upload. This prevents the uploader program from uploading metadata that is invalid with respect to the policy of the Pacifica Ingest server. - When the uploader program is ready to bundle the data, the uploader program
MUST construct a
list
of objects, representing the fields of the corresponding instance of thetar.TarInfo
class. Each object MUST export afileobj
field whose value implements the file protocol, i.e., exports aread()
method. - The uploader program MUST construct a new instance of the
pacifica.uploader.bundler.Bundler
class using the instances of thepacifica.uploader.metadata.MetaUpdate.MetaUpdate
andtar.TarInfo
classes, as previously stated in paragraph 11. Then, the uploader program MUST construct a file-like object that can be written to in binary mode, and then call thepacifica.uploader.bundler.Bundler.stream()
method. - The uploader program MUST construct a new instance of the
pacifica.uploader.Uploader.Uploader
class. Then, the uploader program MUST construct a file-like object that can be read in binary mode, and then call thepacifica.uploader.bundler.Bundler.upload()
method. - Finally, the uploader program MUST verify the result of the ingest by
calling the
pacifica.uploader.Uploader.Uploader.getstate()
method. If an ingest-related error occurs, then the uploader program MAY repeat the ingest operation.