SquirrelsProject

This class is used to interact with a Squirrels project from Python. For example, you can create a SquirrelsProject object in Python (or Jupyter Notebook) as such:

from squirrels import SquirrelsProject

sqrl = SquirrelsProject(project_path="path/to/squirrels/project/")

And then call methods on the SquirrelsProject object to perform various operations. This class can be imported from the squirrels module.

Constructor

Creates a SquirrelsProject object.

def __init__(
    self, *, project_path: str = ".", load_dotenv_globally: bool = False,
    log_to_file: bool = False, 
    log_level: Literal["DEBUG", "INFO", "WARNING"] = "INFO", 
    log_format: Literal["text", "json"] = "text"
) -> None:

Hide arguments

string

default:"."

Path to the Squirrels project folder. Defaults to current working directory.

boolean

default:false

If True, loads variables from .env and .env.local into the process environment for the lifetime of this object.

boolean

default:false

Whether to enable logging to file(s) in the “logs/” folder (or a custom folder). Default is from SQRL_LOGGING__TO_FILE environment variable or false.

string

default:"INFO"

Logging level to include in the log file. One of “DEBUG”, “INFO”, “WARNING”.

string

default:"text"

Format of the log file. One of “text”, “json”.

Methods

Methods that can be invoked from the SquirrelsProject object.

compile()

Async method to compile the SQL templates into files in the “target/” folder. Same functionality as the “sqrl compile” CLI.

async def compile(
    self, *, selected_model: str | None = None, test_set: str | None = None, 
    do_all_test_sets: bool = False, runquery: bool = False, clear: bool = False, 
    buildtime_only: bool = False, runtime_only: bool = False
) -> None:

Hide arguments

string | None

default:"None"

Name of the model to compile. If specified, the compiled SQL is also printed (for SQL models). If None, compiles all models.

string | None

default:"None"

Name of the test set to compile with. If None, uses the default test set (varies by dataset). Ignored if do_all_test_sets is True.

boolean

default:false

Compile all applicable test sets for the selected dataset(s). If True, test_set is ignored.

boolean

default:false

Run all compiled queries and save each result as CSV (runtime only). If True and selected_model is specified, all upstream models of the selected model are compiled as well.

boolean

default:false

Clear the entire target/compile/ folder before compiling.

boolean

default:false

Compile only buildtime (static/build) models.

boolean

default:false

Compile only runtime (dbviews/federates) models.

None

No return value.

get_all_data_models()

Async method to list all data models in the project.

async def get_all_data_models(self) -> list[DataModelItem]:

list[DataModelItem]

A list of DataModelItem objects. The DataModelItem object has the following properties:

Show DataModelItem properties

string

The name of the model.

string

The type of the model.

object

The configuration of the model.

boolean

Whether the model is queryable.

get_all_data_lineage()

Async method to retrieve lineage across models, datasets, and dashboards.

async def get_all_data_lineage(self) -> list[LineageRelation]:

LineageRelation[]

A list of LineageRelation objects. The LineageRelation object has the following properties:

Show LineageRelation properties

Literal['buildtime', 'runtime']

The type of the lineage relation.

LineageNode

The source of the lineage relation. A LineageNode object (defined below).

LineageNode

The target of the lineage relation. A LineageNode object (defined below).

Show LineageNode properties

string

The name of the node.

Literal['model', 'dataset', 'dashboard']

The type of the node.

seed()

Method to retrieve a seed as a polars LazyFrame given a seed name.

def seed(self, name: str) -> polars.LazyFrame:

Hide arguments

string

required

Name of the seed to retrieve.

polars.LazyFrame

A polars LazyFrame of the seed data.

build()

Async method to build the virtual data environment. Same functionality as the “sqrl build” CLI.

async def build(
  self, *, full_refresh: bool = False, select: str | None = None
) -> None:

Hide arguments

boolean

default:false

Drop all tables and rebuild the virtual data environment from scratch.

string | None

default:"None"

Name of a single data model to build; if None, builds all data models.

None

No return value.

dataset_metadata()

Method to retrieve the metadata (descriptions, columns, etc.) for a dataset.

def dataset_metadata(self, name: str) -> DatasetMetadata:

Hide arguments

string

required

Name of the dataset to retrieve metadata for.

DatasetMetadata

A DatasetMetadata object.

dataset_result()

Async method to retrieve the dataset result given parameter selections. Unlike the dataset method of DashboardArgs, this method returns a DatasetResult object (which includes metadata) instead of a polars DataFrame. Furthermore, while the DashboardArgs method automatically borrows selections applied to the dashboard, this method requires selections to be explicitly provided through code.

async def dataset_result(
    self, name: str, *, selections: dict[str, Any] = {}, 
    user: RegisteredUser | None = None, configurables: dict[str, str] = {}
) -> DatasetResult:

Hide arguments

string

required

Name of the dataset to retrieve.

dict[str, Any]

Parameter selections to apply to the dataset.

RegisteredUser | None

default:"None"

RegisteredUser object to use for authentication. If None, a guest user with default values for custom user fields is used.Since this method fetches the dataset through code instead of the API, this user is not used to check access on the scope of the dataset. Instead, this argument is more applicable for applying user attributes to the dataset result.

dict[str, str]

Optional overrides for configurable placeholders defined in your project.

DatasetResult

A DatasetResult object.

dashboard()

Async method to retrieve a dashboard given parameter selections.

async def dashboard(
    self, name: str, *, selections: dict[str, Any] = {}, 
    user: RegisteredUser | None = None, 
    dashboard_type: type[PngDashboard | HtmlDashboard] = PngDashboard, 
    configurables: dict[str, str] = {}
) -> PngDashboard | HtmlDashboard:

Hide arguments

string

required

The name of the dashboard to retrieve.

dict[str, Any]

Parameter selections to apply to the dashboard.

RegisteredUser | None

default:"None"

RegisteredUser object to use for authentication. If None, a guest user with default values for custom user fields is used.

type[PngDashboard | HtmlDashboard]

default:"PngDashboard"

Return type used for type hints (if you rely on them). For example, provide PngDashboard to return a PngDashboard.

dict[str, str]

Optional overrides for configurable placeholders defined in your project.

PngDashboard | HtmlDashboard

A PngDashboard or HtmlDashboard object based on dashboard_type.

query_models()

Async method to query the data models with SQL given parameter selections.

async def query_models(
    self, sql_query: str, *, selections: dict[str, Any] = {}, 
    user: RegisteredUser | None = None, configurables: dict[str, str] = {}
) -> DatasetResult:

Hide arguments

string

required

SQL query to execute.

dict[str, Any]

Parameter selections to apply to the data models.

RegisteredUser | None

default:"None"

RegisteredUser object to use for attributes of authenticated user. If None, a guest user with default values for custom user fields is used.

dict[str, str]

Optional overrides for configurable placeholders defined in your project.

DatasetResult

A DatasetResult object.

get_compiled_model_query()

Async method to compile a specific data model and return its language and compiled definition.

async def get_compiled_model_query(
    self, model_name: str, *, selections: dict[str, Any] = {}, 
    user: RegisteredUser | None = None, configurables: dict[str, str] = {}
) -> CompiledQueryModel:

Hide arguments

string

required

Name of the data model to compile.

dict[str, Any]

Parameter selections to apply during compilation.

RegisteredUser | None

default:"None"

RegisteredUser object to use for attributes of authenticated user. If None, a guest user with default values for custom user fields is used.

dict[str, str]

Optional overrides for configurable placeholders defined in your project.

CompiledQueryModel

A CompiledQueryModel object with language, definition, and placeholders.

get_fastapi_components()

Get the FastAPI components for the Squirrels project including mount path, lifespan, and FastAPI app.

def get_fastapi_components(
    self, *, no_cache: bool = False, host: str | None = None, port: int | None = None, 
    mount_path_format: str = "/analytics/{project_name}/v{project_version}"
) -> FastAPIComponents:

Hide arguments

boolean

default:false

Whether to disable caching for parameter options, datasets, and dashboard results in the API server.

string

default:"localhost"

The host the API server will listen on. Only used for the welcome banner.

integer

default:"8000"

The port the API server will listen on. Only used for the welcome banner.

string

default:"/analytics/{project_name}/v{project_version}"

The format of the mount path. Use and as placeholders.Any underscores (”_”) in the project name are replaced by dash (”-”) for the API path for the placeholder.

FastAPIComponents

A FastAPIComponents object with properties:

mount_path (str): The mount path for the Squirrels project.
lifespan (AsyncGeneratorContextManager): The lifespan context manager for the Squirrels project.
fastapi_app (FastAPI): The FastAPI app for the Squirrels project.

close()

Use this method to deliberately close any database connections opened by the SquirrelsProject object. The database connections may still be opened again by other methods invoked on the SquirrelsProject object after this method is called.

def close(self) -> None:

None

No return value.

Examples

Here are some common usage patterns for the SquirrelsProject class. It is assumed that the code is running in an async context (e.g. inside an async function or a Jupyter Notebook cell).

Basic project initialization

from squirrels import SquirrelsProject

# Initialize with default settings (current directory)
sqrl = SquirrelsProject()

# Initialize with specific project path
sqrl = SquirrelsProject(project_path="path/to/project")

# Initialize with environment variables and file logging
sqrl = SquirrelsProject(
    project_path="path/to/project",
    log_format="json",
    log_level="DEBUG"
)

Compiling and building models

from squirrels import SquirrelsProject

sqrl = SquirrelsProject()

# Compile all models
await sqrl.compile()

# Compile a specific model and print its SQL
await sqrl.compile(selected_model="my_model")

# Compile with a specific test set
await sqrl.compile(test_set="test_set_1")

# Build the virtual data environment
await sqrl.build()

# Full refresh build (drop and rebuild everything)
await sqrl.build(full_refresh=True)

# Build only a specific model
await sqrl.build(select="my_model")

Querying a dataset

from squirrels import SquirrelsProject

sqrl = SquirrelsProject()

# Get dataset metadata
metadata = sqrl.dataset_metadata("sales_data")
print(f"Description: {metadata.target_model_config.description}")
print(f"Columns: {[col.name for col in metadata.target_model_config.columns]}")

# Query a dataset with parameter selections
result = await sqrl.dataset_result(
    "sales_data",
    selections={
        "date_range": ["2024-01-01", "2024-12-31"],
        "region": "north-america"
    }
)

# The result.df attribute is a polars DataFrame
print(result.df.head())

# Convert to pandas if needed
pandas_df = result.df.to_pandas()
print(pandas_df.head())

Working with authentication

from squirrels import SquirrelsProject
from squirrels.auth import RegisteredUser, CustomUserFields

from pyconfigs.user import CustomUserFields

sqrl = SquirrelsProject()

# Note: CustomUserFields must be extended in your project's user.py
custom_fields = CustomUserFields()  # Use your extended class

# Create a registered user with custom fields
user = RegisteredUser(username="john_doe", custom_fields=custom_fields)

# Query dataset as authenticated user
result = await sqrl.dataset_result(
    "protected_data",
    selections={"year": "2024"},
    user=user
)

# Querying protected datasets with code also allows for bypassing authentication
result = await sqrl.dataset_result(
    "protected_data",
    selections={"year": "2024"}
)

# Access the polars DataFrame
print(result.df)

Working with dashboards

Render a PNG dashboard in a Jupyter Notebook:

from squirrels import SquirrelsProject
from squirrels.dashboards import PngDashboard, HtmlDashboard

sqrl = SquirrelsProject()

# Get a PNG dashboard
png_dashboard = await sqrl.dashboard(
    "sales_overview",
    selections={"quarter": "Q1"},
    dashboard_type=PngDashboard
)
png_dashboard

Render an HTML dashboard in a Jupyter Notebook:

from squirrels import SquirrelsProject
from squirrels.dashboards import HtmlDashboard

sqrl = SquirrelsProject()

# Get an HTML dashboard
html_dashboard = await sqrl.dashboard(
    "interactive_report",
    selections={"year": "2024"},
    dashboard_type=HtmlDashboard
)
html_dashboard

Querying models directly

from squirrels import SquirrelsProject

sqrl = SquirrelsProject()

# Query models using custom SQL
result = await sqrl.query_models(
    "SELECT * FROM my_model WHERE amount > 1000",
    selections={"date_param": "2024-01-01"}
)

# Access the polars DataFrame
print(result.df)

# Get compiled query for a specific model
compiled = await sqrl.get_compiled_model_query(
    "my_model",
    selections={"region": "west"}
)

print(f"Language: {compiled.language}")
print(f"Query:\n{compiled.definition}")

Working with seeds

from squirrels import SquirrelsProject

sqrl = SquirrelsProject()

# Get a seed as a polars LazyFrame
seed_lf = sqrl.seed("lookup_table")

# Collect and print the data
df = seed_lf.collect()
print(df)

Exploring project metadata

from squirrels import SquirrelsProject

sqrl = SquirrelsProject()

# List all data models
models = await sqrl.get_all_data_models()
for model in models:
    print(f"{model.name} ({model.model_type}): queryable={model.is_queryable}")

# Get data lineage
lineage = await sqrl.get_all_data_lineage()
for relation in lineage:
    print(f"{relation.source.name} ({relation.source.type}) -> "
          f"{relation.target.name} ({relation.target.type}) [{relation.type}]")

Close database connections

from squirrels import SquirrelsProject

sqrl = SquirrelsProject()

# Perform operations...

# Close database connections associated with the project when done
sqrl.close()

arguments

auth

connections

parameters

parameter_options

data_sources

dashboards

types

Constructor

Methods

compile()

get_all_data_models()

get_all_data_lineage()

seed()

build()

dataset_metadata()

dataset_result()

dashboard()

query_models()

get_compiled_model_query()

get_fastapi_components()

close()

Examples

Basic project initialization

Compiling and building models

Querying a dataset

Working with authentication

Working with dashboards

Querying models directly

Working with seeds

Exploring project metadata

Close database connections

​Constructor

​Methods

​compile()

​get_all_data_models()

​get_all_data_lineage()

​seed()

​build()

​dataset_metadata()

​dataset_result()

​dashboard()

​query_models()

​get_compiled_model_query()

​get_fastapi_components()

​close()

​Examples

​Basic project initialization

​Compiling and building models

​Querying a dataset

​Working with authentication

​Working with dashboards

​Querying models directly

​Working with seeds

​Exploring project metadata

​Close database connections

Constructor

Methods

compile()

get_all_data_models()

get_all_data_lineage()

seed()

build()

dataset_metadata()

dataset_result()

dashboard()

query_models()

get_compiled_model_query()

get_fastapi_components()

close()

Examples

Basic project initialization

Compiling and building models

Querying a dataset

Working with authentication

Working with dashboards

Querying models directly

Working with seeds

Exploring project metadata

Close database connections