Time Cost

We decrease project time-to-value with more reuse of existing code. There’s significant opportunity for more reuse of feature transform pipeline code, especially project-to-project. Code re-writes are common between projects, or even within-project, depending on gaps between R&D and software engineers. But there’s a better way.

Complexity (Technical Debt) Cost

Code complexity slows progress. Code complexity may be due to system design challenges: components lack orthogonal, modular nature. When a code workflow becomes hard to mentally model, it’s very hard to:

• Describe, debug current state
• Deliver improvements with confidence
• Inspire trust in system behavior and results

There’s significant opportunity to de-couple (a) standard transforms workflow from (b) project-specific configuration. Imagine a feature transform pipeline described with key-value pairs, and controlled from the command line.

Data Sample Configuration

When we analysts receive the question, “what data were used in this analysis?” – one configuration file should quickly confirm. After all, data are real, while models are abstractions. A derived feature transform pipeline depends critically upon data sample details:

• On this process run, will we re-derive/re-train the pipeline?
• From what storage source do we extract data?
  • In a database, which tables record inputs (X) & outcomes (Y)?
• What filters subset data which trained this pipeline? Or, what filters subset a new test run’s data?
  • A descriptive title/tag aids understanding
• What outcome do we seek to predict? How should we treat its missing values?
• What columns constitute data attributes: not-for-modeling subject identifiers, useful in exploratory analysis?

To centrally capture those details, we use a fun-to-read file of key-value pairs. An example follows, from baseball card valuation modeling:

is_training_run: true

source:
  storage_type: database
  X: seed.trading_card_transactions_features
  Y: seed.trading_card_transactions_value

filters:
  title: post_202304
  time_field: id
  time_min: 918
  time_max: 2000

filters_train:
  title: thru_202304
  time_field: id
  time_min: 0
  time_max: 917

outcome_definition: 
  title: unit_price
  do_drop_na: true
  fillna_value: 0

dataset_attributes:
  - id
  - name

Feature Transforms Configuration

When we analysts ask the question, “what transformation steps yield my model-ready data?” – one configuration file should quickly confirm. There’s a better way than each time combing through scripts. Moreover, an analyst adds high value by determining smart data transformations. Operations code is more a liability than an asset.

Feature transforms may be sufficiently configured in one file:

• Uniquely name the workflow. (Arbitrarily many workflows could transform a dataset.)
• Declare transform functions (“transformers”) with preset arguments.
• Declare model features. For each,
  • What’s the data type?
  • Which transforms apply?

Another example from baseball card valuation modeling:

title: narrow


transformers:

  impute_numeric:
    strategy: median
    add_indicator: true

  impute_numeric_zero:
    strategy: constant
    fill_value: 0
    add_indicator: false

  impute_numeric_flag:
    strategy: most_frequent
    add_indicator: true


features:

  print_year:
    dtype: float
    transforms:
      impute_numeric:

  is_base:
    dtype: float
    transforms:
      impute_numeric_flag:

  is_serial_numbered:
    dtype: float
    transforms:
      impute_numeric_flag:

  serial_numbered_to:
    dtype: float
    transforms:
      impute_numeric:

  is_rookie_card:
    dtype: float
    transforms:
      impute_numeric_flag:

  is_autographed:
    dtype: float
    transforms:
      impute_numeric_flag: