Skip to content

Module zillion.warehouse

Warehouse

Bases: zillion.field.FieldManagerMixin

class zillion.warehouse.Warehouse(config=None, datasources=None, ds_priority=None, nlp=False)

A reporting warehouse that contains various datasources to run queries against and combine data in report results. The warehouse may contain global definitions for metrics and dimensions, and will also perform integrity checks of any added datasources.

Note that the id, name, and meta attributes will only be populated when the Warehouse is persisted or loaded from a database.

Parameters:

  • config - (dict, str, or buffer, optional) A dict adhering to the WarehouseConfigSchema or a file location to load the config from
  • datasources - (list, optional) A list of DataSources that will make up the warehouse
  • ds_priority - (list, optional) An ordered list of datasource names establishing querying priority. This comes into play when part of a report may be satisfied by multiple datasources. Datasources earlier in this list will be higher priority.
  • nlp - (bool, optional) If true, allow NLP analysis when creating fields
add_datasource(self, ds, skip_integrity_checks=False)

Add a datasource to this warehouse

Parameters:

  • ds - (DataSource) The datasource object to add
  • skip_integrity_checks - (bool, optional) If True, skip warehouse integrity checks
add_dimension(self, dimension, force=False)

Add a reference to a dimension to this FieldManager

add_metric(self, metric, force=False)

Add a reference to a metric to this FieldManager

apply_config(self, config, skip_integrity_checks=False, nlp=False)

Apply a warehouse config

Parameters:

  • config - (dict) A dict adhering to the WarehouseConfigSchema
  • skip_integrity_checks - (bool, optional) If True, skip warehouse integrity checks
  • nlp - (bool, optional) If true, allow NLP analysis when creating fields
delete(id)

Delete a saved warehouse. Note that this does not delete any report specs that reference this warehouse ID.

Parameters:

  • id - (int) The ID of a Warehouse to delete
delete_report(self, spec_id)

Delete a report by spec ID

Parameters:

  • spec_id - (int) The ID of a report spec to delete
directly_has_dimension(self, name)

Check if this FieldManager directly stores this dimension

directly_has_field(self, name)

Check if this FieldManager directly stores this field

directly_has_metric(self, name)

Check if this FieldManager directly stores this metric

execute(self, metrics=None, dimensions=None, criteria=None, row_filters=None, rollup=None, pivot=None, order_by=None, limit=None, limit_first=False, adhoc_datasources=None, allow_partial=False)

Build and execute a Report

Returns:

(ReportResult) - The result of the report

execute_id(self, spec_id, adhoc_datasources=None)

Build and execute a report from a spec ID

Parameters:

  • spec_id - (int) The ID of a report spec
  • adhoc_datasources - (list, optional) A list of FieldManagers specific to this request

Returns:

(ReportResult) - The result of the report

execute_text(self, text, adhoc_datasources=None, allow_partial=False)

Build and execute a report from a natural language query. Requires the nlp extension to be installed.

Parameters:

  • text - (str) A natural language query
  • adhoc_datasources - (list, optional) A list of FieldManagers specific to this request
  • allow_partial - (bool, optional) If True, allow partial

Returns:

(ReportResult) - The result of the report

from_data_file(*args, **kwargs)

Pass args through to DataSource.from_data_file and create a Warehouse

from_db_file(*args, **kwargs)

Pass args through to DataSource.from_db_file and create a Warehouse

get_child_field_managers(self)

Get a list of all datasources in this warehouse

get_datasource(self, name, adhoc_datasources=None)

Get the datasource object corresponding to this datasource name

Parameters:

  • name - (str) The name of the datasource
  • adhoc_datasources - (list, optional) A list of FieldManagers specific to this request

Returns:

(DataSource) - The matching datasource object

get_dimension(self, obj, adhoc_fms=None)

Get a reference to a dimension on this FieldManager

get_dimension_configs(self, adhoc_fms=None)

Get a dict of all dimension configs supported by this FieldManager

get_dimension_names(self, adhoc_fms=None)

Get a set of dimension names supported by this FieldManager

get_dimension_table_set(self, grain, dimension_grain, adhoc_datasources=None)

Get a TableSet that can satisfy dimension table joins across this grain

Parameters:

  • grain - (list) A list of dimension names representing the full grain required including dimension and criteria grain
  • dimension_grain - (list of str) A list of dimension names representing the requested dimensions for report grouping
  • adhoc_datasources - (list, optional) A list of FieldManagers for this request

Returns:

(TableSet) - A TableSet that can satisfy this request

get_dimensions(self, adhoc_fms=None)

Get a dict of all dimensions supported by this FieldManager

get_direct_dimension_configs(self)

Get a dict of dimension configs directly supported by this FieldManager

get_direct_dimensions(self)

Get dimensions directly stored on this FieldManager

get_direct_fields(self)

Get a dict of all fields directly supported by this FieldManager

get_direct_metric_configs(self)

Get a dict of metric configs directly supported by this FieldManager

get_direct_metrics(self)

Get metrics directly stored on this FieldManager

get_field(self, obj, adhoc_fms=None)

Get a reference to a field on this FieldManager

get_field_instances(self, field, adhoc_fms=None)

Get a dict of FieldManagers (including child and adhoc FMs) that support a field

get_field_managers(self, adhoc_fms=None)

Get a list of all child FieldManagers including adhoc

get_field_names(self, adhoc_fms=None)

Get a set of field names supported by this FieldManager

get_fields(self, adhoc_fms=None)

Get a dict of all fields supported by this FieldManager

get_metric(self, obj, adhoc_fms=None)

Get a reference to a metric on this FieldManager. If the object passed is a dict it is expected to define an AdHocMetric.

get_metric_configs(self, adhoc_fms=None)

Get a dict of all metric configs supported by this FieldManager

get_metric_names(self, adhoc_fms=None)

Get a set of metric names supported by this FieldManager

get_metric_table_set(self, metric, grain, dimension_grain, adhoc_datasources=None)

Get a TableSet that can satisfy a metric at a given grain

Parameters:

  • metric - (str) A metric name
  • grain - (list) A list of dimension names representing the full grain required including dimension and criteria grain
  • dimension_grain - (list of str) A list of dimension names representing the requested dimensions for report grouping
  • adhoc_datasources - (list, optional) A list of FieldManagers for this request

Returns:

(TableSet) - A TableSet that can satisfy this request

get_metrics(self, adhoc_fms=None)

Get a dict of all metrics supported by this FieldManager

has_dimension(self, name, adhoc_fms=None)

Check whether a dimension is contained in this FieldManager

has_field(self, name, adhoc_fms=None)

Check whether a field is contained in this FieldManager

has_metric(self, name, adhoc_fms=None)

Check whether a metric is contained in this FieldManager

init_embeddings(self, force_recreate=False)

Initialize the warehouse embeddings collection. This is necessary to use natural language query to report features.

Parameters:

  • force_recreate - (bool, optional) If True, force the embeddings collection to be recreated from scratch.
load(id)

Load a Warehouse from a Warehouse ID

Parameters:

  • id - (int) A Warehouse ID

Returns:

(Warehouse) - A Warehouse object

load_report(self, spec_id, adhoc_datasources=None)

Load a report from a spec ID

Parameters:

  • spec_id - (int) The ID of a report spec
  • adhoc_datasources - (list, optional) A list of FieldManagers specific to this request

Returns:

(Report) - A report built from this report spec

load_report_and_warehouse(spec_id)

Load a Report and Warehouse from a ReportSpec. The Warehouse will be populated on the returned Report object.

Parameters:

  • spec_id - (int) A ReportSpec ID

Returns:

(Report) - A Report built from this report spec

load_warehouse_for_report(spec_id)

Load the warehouse corresponding to the ReportSpec

Parameters:

  • spec_id - (int) A ReportSpec ID

Returns:

(Warehouse) - A Warehouse object

print_dimensions(self, indent=None)

Print all dimensions in this FieldManager

print_info(self)

Print the warehouse structure

print_metrics(self, indent=None)

Print all metrics in this FieldManager

remove_datasource(self, ds, skip_integrity_checks=False)

Remove a datasource from this config

Parameters:

  • ds - (DataSource) The datasource object to remove
  • skip_integrity_checks - (bool, optional) If True, skip warehouse integrity checks
run_integrity_checks(self, adhoc_datasources=None)

Run a series of integrity checks on the warehouse and its datasources. This will raise a WarehouseException with all failed checks.

Parameters:

  • adhoc_datasources - (list, optional) A list of FieldManagers to include for this request
save(self, name, config_url, meta=None)

Save the warehouse config and return the ID

Parameters:

  • name - (str) A name to give the Warehouse
  • config_url - (str) A URL pointing to a config file that can be used to recreate the warehouse
  • meta - (object, optional) A metadata object to be serialized as JSON and stored with the warehouse. This will be merged into existing meta settings if present and take precedence.

Returns:

(int) - The ID of the saved Warehouse

save_report(self, meta=None, **kwargs)

Init a Report and save it as a ReportSpec. Note that the Warehouse must be saved before any ReportSpecs can be saved for the Warehouse.

Parameters:

  • meta - (object, optional) A metadata object to be serialized as JSON and stored with the report
  • **kwargs - Passed through to Report

Returns:

(Report) - The built report with the spec ID populated