Skip to main content

Data

Vom has a Feature Store built-in. It is a time series database of features that can be used for decision making. Data can be pushed into features by customers or pulled from connectors, such as credit scores being pulled from a credit bureau.

Features​

Features are a way to feed data into Vom for making decisions. Customers can push data into features, or features can pull data from connectors, such as credit scores from a credit bureau.

A feature is about an entity, like a customer, and has a series of values over time. For example, John Borrower had a credit score of 550 on Feb 1st and 590 on Mar 2nd.

Here are the settings that must be configured for each feature:

  • Name: How the feature is referred to when pushing data into it or reading it when executing Policy Versions.
  • Keys: The feature is about an entity, which is identified by one or more keys. For example, a credit_score feature might be keyed by SSN, and a loan_amount feature by both SSN and a customer-defined Loan ID. The latter would be both about a loan and the person who took it. When pushing data into a feature, values must be provided for all its keys, so that Vom knows which entities the data is about.
  • Data Type: The values of a feature must be all of the same type, and setting this type allows the values to be used only in the right context. For example, it makes sense to divide numbers, but not texts. The types supported by Vom features are:
    • Integer
    • Decimal (AKA Floating Point)
    • Boolean (True/False)
    • Text (AKA String)
    • Datetime: Instant in time, including the date and time of day. Values must be provided as a combined date and time ISO 8601 string in the UTC timezone (i.e., ending with Z), for example 2024-01-20T14:00:00Z.

Features can be created from scratch or indirectly by configuring connectors that feed those features. In the former case, data has to be pushed into features using the Feature Value Pushing API.

Feature Value Pushing API​

Feature values can be pushed into Vom one at a time or in batch. Batch pushing is strongly preferable if you have many values to push, as it's much faster. Sending values one at a time is a good option if you want to stream data into Vom as it gets created. For example, you might want to tell Vom of a loan immediately after it has been granted, so that fact can be used in new decisions.

For large datasets, we recommend splitting the data into chunks of around 2,000 records each and sending each chunk using one batch API call.

Here are some important error conditions, their response codes, and how to handle them when pushing feature values:

  • Permanent failures should be counted as failed decisions and logged1 for debugging:
    • 400: An invalid request was sent. For example, non-numeric text was sent when an integer was expected.
    • 412: There is a missing prerequisite configuration. For example a data push was attempted to a nonexistent feature.
  • 500, 429: Transient failures which should be retried with exponential backoff or aborted depending on time sensitivity. Count these errors but logging is optional as the content is usually not informative.
  • Other errors are typically non-transient and rare. Do not retry these; instead, log the JSON result for debugging.

See the detailed API documentation at the swagger.

For batch operations, we recommend pushing features daily. To ensure data consistency, verify each daily push succeeds through automatic or manual retries.

Connectors​

A connector is a way to pull data from an external source into Vom. For example a connector can pull credit scores from a credit bureau. Connectors are invoked during decisions at the time a policy version needs to read a feature value, and no value fresh enough exists in the feature store. That allows saving on costs by:

  • Configuring policies to use cached feature values, by setting the Value Expiration option on the corresponding input variable.
  • Structuring the policy to only read expensive features after simpler checks have passed. Notice that if a feature is only read on a block that never runs, its connector will not be invoked.

HTTP Status Matchers​

HTTP connectors support status matchers. A status matcher evaluates a JMESPath condition against the response body and, when it is truthy, replaces the response status code with the configured one.

This is useful when providers return business errors in the response body while still using 2xx HTTP statuses.

In debug/execution details, Vom stores both:

  • original_status: the raw provider status code;
  • status_code: the final status after status matcher evaluation.

JMESPath​

The HTTP connector can extract arbitrary data from JSON Responses using using JMESPath2 expressions, such as data.addresses[0].city.

If the provider returns XML, Vom converts the response to JSON before running extractors. Write JMESPath expressions against the converted JSON structure, not the original XML text.

Apart from the JMESPath built-in functions, the HTTP Connector JMESPath also supports the following functions to make common tasks easy:

parse_datetime​

Convert from any date format to Vom's date format.

All datetimes Feature values are expected to be in combined date and time ISO 8601 string format in the UTC timezone. However, not all providers return datetimes in this format, so the parse_datetime function can be used to convert them to the expected format.

Signature: parse_datetime(datetime_string, date_format, timezone_identifier)

Parameters:

  • datetime_string: The datetime string to parse or the path to it in the response JSON;
  • date_format: The format of the datetime string to parse using the strftime format;
  • timezone_identifier: The timezone to interpret the datetime in. See the list of timezone identifiers. Usually this would the official timezone of the Vom customer, such as the timezone of its headquarters.

Example:

Given the following response:

{
  "user": {
    "name": "John Doe",
    "birthdate": "20/06/1990"
  }
}

The JMESPath expression parse_datetime(user.birthdate, '%d/%m/%Y', 'America/Sao_Paulo') will return the datetime string 1990-06-20T03:00:00Z. That is now in the expected format and can be saved as a Feature value and used within a Policy Version.

parse_float​

Converts a string representation of a number into a float.

All float Feature values are expected to be already a number or a string that uses a dot as the decimal separator (e.g., "1,234.56"). However, not all providers return numbers in this format, so the parse_float function can be used to convert them to the expected format.

Signature: parse_float(value, locale)

Parameters:

  • value: The numerical value or string to parse or the path to it in the response JSON;
  • locale_str: The locale to interpret the number formatting. See the list of locale identifiers.

Example:

Given the following response:

{
  "transaction": {
    "amount": "1.234,56"
  }
}

The JMESPath expression parse_float(transaction.amount, 'pt_BR') will return 1234.56. That is now in the expected format and can be saved as a Feature value and used within a Policy Version.

lower​

Converts a string to lowercase.

Signature: lower(string)

Parameters:

  • string: The string to convert to lowercase or the path to it in the response JSON.

Example:

Given the following response:

{
  "user": {
    "name": "John Doe"
  }
}

The JMESPath expression lower(user.name) will return "john doe".

Returns the first match of a regular expression (regex) pattern in a string.

The regex_search function is useful when you need to extract part of a string based on a specific pattern. It returns the first match found in the string.

Signature: regex_search(pattern, string)

Parameters:

  • pattern: The regex pattern to search for. If null, the function returns null;
  • string: The string in which to search for the pattern or the path to it in the response JSON.

Example:

Given the following response:

{
  "user": {
    "bio": "Group #42."
  }
}

The JMESPath expression regex_search('#\d+', user.bio) will return "42".

filter_strings_by_substrings​

Returns all strings from a list that contain any of the given substrings ignoring case and diacritics.

This function is useful when you need to filter a list of strings to only those that include at least one match from a list of substrings. Matching is case-insensitive and diacritic-insensitive, making it effective in multilingual or inconsistently formatted strings.

Signature: filter_strings_by_substrings(strings, substrings)

Parameters:

  • strings: The list of strings to filter or the path to it in the response JSON;
  • substrings: The list of substrings to check for in each string.

Example:

Given the following response:

{
  "names": [
    "François Dupont",
    "Jose Almeida",
    "João Pereira",
    "Chloé Martin",
    "Maria Souza"
  ]
}

The JMESPath expression filter_strings_by_substrings(names, ['francois', 'joao', 'chloe']) will return ["François Dupont", "João Pereira", "Chloé Martin"] hence the JMESPath expression length(filter_strings_by_substrings(names, ['francois', 'joao', 'chloe'])) will return 3.

now​

Returns the current date and time in ISO 8601 format in the UTC timezone.

Signature: now()

Example:

Considering that now is noon on December 31th 2024 in the UTC timezone.

The JMESPath expression now() will return "2024-12-31T12:00:00Z".

days​

Returns the difference between two ISO 8601 date strings in days. If either parameter misses timezone information, the function will fail.

Signature: days(start_date, end_date)

Parameters:

  • start_date: The start date or the path to it in the response JSON;
  • end_date: The end date or the path to it in the response JSON.

Example:

Given the following response:

{
  "start_date": "2024-12-31T00:00:00Z",
  "end_date": "2025-01-01T00:00:00Z"
}

The JMESPath expression days(start_date, end_date) will return 1.

gap_months_until_now​

Returns the number of months not covered by any period between a specified number of years ago and today.

It’s useful for finding gaps in time, such as periods without work, activity, contracts, or subscriptions.

Signature: gap_months_until_now(start_dates, end_dates, years_offset)

  • start_dates: A list of ISO 8601 datetime strings marking when each period starts;
  • end_dates: A list of ISO 8601 datetime strings marking when each period ends;
  • years_offset: The number of years in the past to start counting from.
info

start_dates and end_dates must have the same length and order.
Each start date is paired with the end date at the same position in the list.

Example:

Given the following response:

{
  "active_months": [
    {
      "name": "May",
      "start_date": "2025-05-01T00:00:00Z",
      "end_date": "2025-05-31T23:59:59Z"
    },
   {
      "name": "July",
      "start_date": "2025-07-01T00:00:00Z",
      "end_date": "2025-07-31T23:59:59Z"
    },
    {
      "name": "October",
      "start_date": "2025-10-01T00:00:00Z",
      "end_date": "2025-10-31T23:59:59Z"
    }
  ]
}

Considering today as January 1st 2026, the JMESPath expression gap_months_until_now(active_months[].start_date, active_months[].end_date, `1`) will return will return 9 since only 3 months in the 1 year period were active.

Footnotes​

  1. The field error in the response JSON has all the information about the error and can be logged. ↩

  2. JMESPath is a query language for JSON. ↩