Feast

Important Capabilities

Capability	Status	Notes
Descriptions	✅	Enabled by default
Schema Metadata	✅	Enabled by default
Table-Level Lineage	✅	Enabled by default

This plugin extracts:

• Entities as MLPrimaryKey
• Fields as MLFeature
• Feature views and on-demand feature views as MLFeatureTable
• Batch and stream source details as Dataset
• Column types associated with each entity and feature

CLI based Ingestion

Starter Recipe

Check out the following recipe to get started with ingestion! See below for full configuration options.

For general pointers on writing and running a recipe, see our main recipe guide.

source:
  type: feast
  config:
    # Coordinates
    path: "/path/to/repository/"
    # Options
    environment: "PROD"

sink:
  # sink configs

Config Details

Options

Note that a . is used to denote nested fields in the YAML recipe.

Field	Description
path ✅ string	Path to Feast repository
enable_owner_extraction boolean	If this is disabled, then we NEVER try to map owners. If this is enabled, then owner_mappings is REQUIRED to extract ownership. Default: False
enable_tag_extraction boolean	If this is disabled, then we NEVER try to extract tags. Default: False
environment string	Environment to use when constructing URNs Default: PROD
fs_yaml_file string	Path to the feature_store.yaml file used to configure the feature store
owner_mappings array	Mapping of owner names to owner types
owner_mappings.map map(str,string)

Schema

The JSONSchema for this configuration is inlined below.

{
  "title": "FeastRepositorySourceConfig",
  "type": "object",
  "properties": {
    "path": {
      "title": "Path",
      "description": "Path to Feast repository",
      "type": "string"
    },
    "fs_yaml_file": {
      "title": "Fs Yaml File",
      "description": "Path to the `feature_store.yaml` file used to configure the feature store",
      "type": "string"
    },
    "environment": {
      "title": "Environment",
      "description": "Environment to use when constructing URNs",
      "default": "PROD",
      "type": "string"
    },
    "owner_mappings": {
      "title": "Owner Mappings",
      "description": "Mapping of owner names to owner types",
      "type": "array",
      "items": {
        "type": "object",
        "additionalProperties": {
          "type": "string"
        }
      }
    },
    "enable_owner_extraction": {
      "title": "Enable Owner Extraction",
      "description": "If this is disabled, then we NEVER try to map owners. If this is enabled, then owner_mappings is REQUIRED to extract ownership.",
      "default": false,
      "type": "boolean"
    },
    "enable_tag_extraction": {
      "title": "Enable Tag Extraction",
      "description": "If this is disabled, then we NEVER try to extract tags.",
      "default": false,
      "type": "boolean"
    }
  },
  "required": [
    "path"
  ],
  "additionalProperties": false
}

Code Coordinates

• Class Name: datahub.ingestion.source.feast.FeastRepositorySource
• Browse on GitHub

Questions

If you've got any questions on configuring ingestion for Feast, feel free to ping us on our Slack.