Usage Guide¶
The Saturnin package contains the basic runtime environment for Firebird Butler services, and forms the basis of the Python implementation of the Firebird Butler platform. The Butler services themselves are not part of the base distribution, and packages with these services must be installed separately. The basic set of services for Saturnin is distributed as saturnin-core package.
The basic concepts on which Saturnin is built are described in introduction to Firebird Butler.
For instructions how to install and setup Saturnin on your machine, see the Getting Started guide.
Basic Architecture¶
Saturnin can be divided into two main parts:
Basic modules for implementing Butler services in Python. This documentation provides only the reference guide for these modules. Instructions (and examples) for their use for creating Butler services for the Saturnin platform are part of the documentation for the saturnin-sdk package.
Tools for managing and running Butler services, and their composition into functional units. This user guide describes this part of the platform.
Butler Services¶
Saturnin currently supports two types of Butler services:
Standard Butler services. These services use the FBSP protocol and work in client/server mode.
Microservices. These are primarily data processing services that use the FBDP protocol for communication within the data processing pipeline.
Butler services are implemented as Python classes in Saturnin. Service properties are
defined by a standard ServiceDescriptor structure, which is registered during installation
using a standard entry point in saturnin.service group.
Service containers¶
Because services are classes, they cannot be invoked directly. Saturnin provides special containers / utilities for running services:
The
saturnin-bundletool provides an environment for running multiple services in separate threads. It can be used both for running related services (eg data processing pipelines) and for centralized operation of unrelated services.The
saturnin-servicetool allows you to run only one service, but allows that service to run either in the main process thread or in a separate thread.
Service containers can also be run as daemons using the saturnin-daemon tool.
Any number of such service container processes can be started. The definition of parameters of the runtime environment, the list of services to be executed and their configuration is described in a special configuration file called Saturnin recipe.
Although it is possible to use these containers directly, Saturnin offers a much more convenient way for normal operation in the form of integrating recipes directly into the saturnin console.
Saturnin recipes¶
Saturnin uses configuration objects based on firebird.base.config module. These objects
could be initialized from / persisted to text files with classic .INI file structure.
Single configuration file could be used to initialize multiple configuration objects, and
Saturnin always uses ConfigParser with extended interpolation support to
process these files.
; this is a comment
# this is also comment
[section]
parameter1 = value
parameter2 =
this is
multiline
value
[another.section]
parameter1 = value
; interpolation for value in the same section
parameter2 = ${parameter1}
; interpolation for value in different section
parameter3 = ${section:parameter1}
Saturnin recipes contain configuration information that describes:
Recipe parameters (
SaturninRecipe)Container configuration. Used section(s), their name(s) and structure depend on used container. By default:
the
saturnin-bundlecontainer uses a “bundle” section with aServiceBundleConfigstructure that refers to the configuration sections of individual components.the
saturnin-servicecontainer uses a “service” section with aServiceExecConfigstructure. The same section is then used to configure the running component.
Configuration of components (sections used to set up particular component configuration objects)
Optional
loggingconfigurationOptional
traceconfiguration
Here is a sample recipe to print Firebird log on screen using two Saturnin microservices:
; 1. Recipe parameters
[saturnin.recipe]
recipe_type = bundle
execution_mode = normal
description = Simple recipe that print log from local Firebird server.
; 2. Bundle content
[bundle]
agents = from-server, writer
; Helper section to centralize definition of shared parameters
[pipe]
name = pipe-1
address = inproc://pipe-1
; 3. Confguration of components
[from-server]
agent = 212657dc-2618-5f4b-a8f5-d8d42e99fe7e
pipe = ${pipe:name}
pipe_address = ${pipe:address}
pipe_mode = bind
pipe_format = text/plain;charset=utf-8
server = local
[writer]
agent = 4e606fdf-3fa9-5d18-a714-9448a8085aab
pipe = ${pipe:name}
pipe_address = ${pipe:address}
pipe_mode = connect
pipe_format = text/plain;charset=utf-8
filename = stdout
file_format = text/plain;charset=utf-8
file_mode = write
Saturnin applications¶
Recipes are primarily clearly defined and static. The Saturnin application mechanism is available for the implementation of dynamic recipes. These are specific user-defined commands for the saturnin console that can be associated with specific recipes. Instructions for creating applications, including examples, can be found in the Saturnin SDK documentation.
Saturnin console¶
The saturnin tool is used to manage the Saturnin platform installation. It can be operated
in two modes:
Single command (direct) mode. The required command and parameters are entered directly on the command line, and after the command is executed, the tool is terminated.
Interactive console mode activated by running the tool without additional parameters. The interactive console offers an enhanced command line with persistent command history, command and parameter completion, and interactive help.
Note
The command set available in console mode differs from command set available in direct mode, as some commands (typically those required to run only once or not very often like initialize or create home) are available only in direct mode.
For normal work, we recommend using the interactive console mode. In the following sections, all the commands described are entered in the interactive console.
Saturnin environment¶
Saturnin uses a number of files and directories whose location in the file system corresponds
to the standards for the platform on which it is installed. This basic directory placement
scheme can be changed by using the SATURNIN_HOME environment variable, which sets the root
of the other directory locations. Alternatively, you can create a "home" subdirectory in
the root directory of the virtual environment in which Saturnin is isolated.
Important
Because on Linux or MacOS the default location of some directories may require higher than normal access rights, we recommend that you always use the home directory setting on these platforms.
To set the home directory in the virtual environment (recommended when installing with pipx), use the command:
> saturnin create home
Saturnin directories and configuration files are created with:
> saturnin initialize
Note
It is safe to run initialize on an already initialized environment because existing
directories or configuration files are not overwritten by default.
Run saturnin initialize --help for a complete description of the command and available options.
Directories¶
saturnin console provides list directories command, that prints all directories used by
Saturnin. An existence check is performed, and status of each directory is indicated with ✔ (exists)
and ✖ (missing) marks.
> list directories ╭─ Saturnin directories ──────────────────────────────────────────────────────────────────────────────╮ │ SATURNIN_HOME is set to : /home/saturnin/home │ │ Saturnin configuration ✔ /home/saturnin/home/config │ │ Saturnin data ✔ /home/saturnin/home/data │ │ Run-time data ✔ /home/saturnin/home/run_data │ │ Log files ✔ /home/saturnin/home/logs │ │ Temporary files ✔ /var/tmp/saturnin │ │ Cache ✔ /home/saturnin/home/cache │ │ User-specific configuration ✔ /home/user/.config/saturnin │ │ User-specific data ✔ /home/user/.local/share/saturnin │ │ PID files ✔ /home/saturnin/home/run_data/pids │ │ Recipes ✔ /home/saturnin/home/data/recipes │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
If any directory is listed as missing, it’s necessary to run saturnin initialize command.
Configuration files¶
The saturnin console provides several configuration-related commands:
Command
list configsprints all configuration files used by Saturnin. An existence check is performed, and status of each file is indicated with✔(exists) and✖(missing) marks.> list configs ╭─ Configuration files ───────────────────────────────────────────────────────────────────────────────╮ │ Main configuration ✔ /home/job/python/projects/saturnin/home/config/saturnin.conf │ │ User configuration ✔ /home/pcisar/.config/saturnin/saturnin.conf │ │ Console theme ✔ /home/job/python/projects/saturnin/home/config/theme.conf │ │ Firebird configuration ✔ /home/job/python/projects/saturnin/home/config/firebird.conf │ │ Logging configuration ✔ /home/job/python/projects/saturnin/home/config/logging.conf │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
Command
create configto create particular configuration file with default content.While all required configuration files are created by
saturnin initializecommand, optional files could (or must) be created with this command. This command could be also used to quickly reset any configuration file to default values.Usage:
> create config [OPTIONS] {main|user|firebird|logging|theme} Creates configuration file with default content. ╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────╮ │ * config_file {main|user|firebird|logging|theme} Configuration file to be created │ │ [default: None] │ │ [required] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯ ╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────╮ │ --new-config Create configuration file even if it already exist. │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯Important
The Firebird configuration file is optional as Saturnin is not stricly bound to Firebird RDBMS, and this configuration is needed only when you want to use Firebird-related services. To create this file, the firebird-driver must be installed.
Note
The logging configuration is optional. When defined, it’s automatically used by all Saturnin containers. To use per-container logging configuration, it’s necessary to use the
--conifgcontainer option with separate logging configuration file.Command
show configto show content of particular configuration file.Usage:
> show config [OPTIONS] {main|user|firebird|logging|theme} Show content of configuration file. ╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────╮ │ * config_file {main|user|firebird|logging|theme} Configuration file │ │ [default: None] │ │ [required] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯Command
edit configto edit configuration file content with prefered editor (uses EDITOR environment variable oreditorparameter ofmainoruserconfiguration file).Usage:
> edit config [OPTIONS] {main|user|firebird|logging|theme} Edit configuration file. ╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────╮ │ * config_file {main|user|firebird|logging|theme} Configuration file │ │ [default: None] │ │ [required] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
Data files¶
Saturnin uses number of data files:
Registry of installed services.
Registry of installed applications.
Registry of OIDs.
Saturnin console command history.
Default log file.
The saturnin console command list datafiles prints paths to all data files used by Saturnin. An existence
check is performed, and status of each file is indicated with ✔ (exists) and ✖ (missing) marks.
> list datafiles ╭─ Saturnin data files ───────────────────────────────────────────────────────────────────────────────╮ │ Installed services ✔ /home/home/data/services.toml │ │ Installed applications ✔ /home/home/data/apps.toml │ │ Registered OIDs ✔ /home/home/data/oids.toml │ │ Console history ✔ /home/home/data/saturnin.hist │ │ Default log file ✔ /home/home/logs/saturnin.log │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
Note
All these files may not exist when Saturnin is installed and initialized.
Service and application registries are updated automatically when packages are (un)installed by the Saturnin console. If these registries are not synchronized with installed services and applications (for example, when developing your services), they must be updated separately with the
update registrycommand.The OID registry is optional enhancement, and not necessary for Saturnin operation.
The console command history is managed automatically.
The default log file is optional, as it’s used by default logging (also optional) configuration.
Managing Saturnin packages¶
Saturnin Butler services, Applications and other extensions must be installed into Saturnin virtual environment before they could be used. Saturnin uses standard pip utility to manage Python packages, and the console provides several commands that use the pip version installed in Saturnin virtual environment.
Warning
While you can call the pip from this virtual environment directly, it’s strongly recommended to use Saturnin console commands, as they ensure that Saturnin registries are in sync with installed packages.
Command
install package.Installs Python package into Saturnin virtual environment via
pip.Note
This command is used also to upgrade installed packages using
-Uor--upgradeoption.Usage:
install package [options] <requirement specifier> [package-index-options] ... install package [options] -r <requirements file> [package-index-options] ... install package [options] [-e] <vcs project url> ... install package [options] [-e] <local project path> ... install package [options] <archive url/path> ...
To list all available options, use
?install packageorinstall package --help.Command
uninstall package.Uninstalls Python package from Saturnin virtual environment via
pip.Usage:
uninstall package [options] <package> ... uninstall package [options] -r <requirements file> ...
To list all available options, use
?uninstall packageoruninstall package --help.Note
This command invokes
pip uninstallcommand with implicit--yesparameter, so you’re not asked for confirmation.Command
pip.Runs
pippackage manager in Saturnin virtual environment.Usage:
pip <command> [options]
To list all available commands and options, use
?piporpip --help.Command
list packageslists installed distribution packages with Saturnin components.Note
This command does not list ALL packages installed in Saturnin virtual environment, but only those that contain registered Saturnin components. To list all installed Python packages, use:
pip listcommand.Example:
> list packages Installed Saturnin packages ╭────────────────────────────────┬─────────╮ │ Package │ Version │ ├────────────────────────────────┼─────────┤ │ saturnin-example-dummy │ 0.1.0 │ │ saturnin-core │ 0.8.0 │ │ saturnin-example-app-dummy │ 0.1.0 │ │ saturnin-example-textio │ 0.1.0 │ │ saturnin-example-app-printfile │ 0.1.0 │ │ saturnin-example-roman │ 0.2.0 │ ╰────────────────────────────────┴─────────╯Command
list services.Lists installed Saturnin services.
Usage:
list services [OPTIONS] ╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────╮ │ --with-name TEXT List only services with this string in name │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
Example:
> list services Registered services ╭─────────────────────────────────┬─────────┬────────────────────────────────────────────────╮ │ Service │ Version │ Description │ ├─────────────────────────────────┼─────────┼────────────────────────────────────────────────┤ │ saturnin.binary.reader │ 0.1.1 │ Binary data reader microservice │ │ saturnin.binary.writer │ 0.1.1 │ Binary data writer microservice │ │ saturnin.example.roman │ 0.2.0 │ Sample ROMAN service │ │ saturnin.example.textio │ 0.1.0 │ Sample TEXTIO microservice │ │ saturnin.firebird.log.fromsrv │ 0.2.1 │ Firebird log from server provider microservice │ │ saturnin.firebird.log.parser │ 0.2.1 │ Firebird log parser microservice │ │ saturnin.firebird.trace.parser │ 0.1.1 │ Firebird trace parser microservice │ │ saturnin.firebird.trace.session │ 0.1.1 │ Firebird trace session provider microservice │ │ saturnin.micro.dummy │ 0.1.0 │ Test dummy microservice │ │ saturnin.proto.aggregator │ 0.2.1 │ Protobuf data aggregator microservice │ │ saturnin.proto.filter │ 0.2.1 │ Protobuf data filter microservice │ │ saturnin.proto.printer │ 0.2.1 │ Protobuf data printer microservice │ │ saturnin.text.linefilter │ 0.2.1 │ Text line filter microservice │ │ saturnin.text.reader │ 0.2.1 │ Text reader microservice │ │ saturnin.text.writer │ 0.2.1 │ Text writer microservice │ ╰─────────────────────────────────┴─────────┴────────────────────────────────────────────────╯Command
list applications.Lists installed Saturnin applications.
Usage:
list applications [OPTIONS] ╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────╮ │ --with-name TEXT List only applications with this string in name │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
Example:
> list applications Registered applications ╭─────────────────────────┬─────────┬───────────┬─────────────────────────────╮ │ Application │ Version │ Used │ Description │ ├─────────────────────────┼─────────┼───────────┼─────────────────────────────┤ │ saturnin.app.dummy │ 0.1.0 │ ✖ │ Test dummy application │ │ saturnin.app.print_file │ 0.1.0 │ ✔ │ Print text file application │ ╰─────────────────────────┴─────────┴───────────┴─────────────────────────────╯Note
Applications that are used in registered recipes are marked as
Used.Command
show service.Show information about installed service.
Usage:
show service [SERVICE_ID] ╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────╮ │ service_id [SERVICE_ID] Service UID or name │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
Example:
> show service saturnin.binary.reader UID: 3db461de-f32e-5514-910d-7d021a2436a5 Name: saturnin.binary.reader Version: 0.1.1 Vendor: The Firebird Project Classification: binary/reader Description: Binary data reader microservice Facilities: API: Distribution: saturnin-core
Note
The
Vendorattribute normally displays the UUID of the vendor. If UUID is found in the OID registry, the OID name is displayed instead.Command
show application.Show information about installed application.
Usage:
show application [APP_ID] ╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────╮ │ app_id [APP_ID] Application UID or name │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯ Example:: > show application saturnin.app.print_file UID: c4aa5f0b-74b7-55ea-8fc8-e3eb41335049 Name: saturnin.app.print_file Version: 0.1.0 Vendor: The Firebird Project Classification: text/print Description: Print text file application Distribution: saturnin-example-app-printfile
Note
The
Vendorattribute normally displays the UUID of the vendor. If UUID is found in the OID registry, the OID name is displayed instead.Command
update registry.Updates registry of installed Saturnin components.
The registry is updated automatically when Saturnin packages are manipulated with built-in
install,uninstallorpipcommands. Manual update is required only when packages are added/updated/removed in differet way.Usage:
update registry
Important
Commands list packages, list services, list applications, show service and show application work
with Saturnin registries. They will not provide accurate information if these registries are not synchronized
with installed packages (see update registry command).
OID registry¶
The Firebird Butler spec recommends using UUID for identification purposes, and Saturnin follows that recommendation. Since standard UUIDs are not very suitable for ordinary users, UUIDs derived from the much more understandable OID are used.
Specifically, Saturnin uses version 5 UUIDs - UUIDs based on a SHA-1 hash of a namespace identifier (which is a UUID) and a name (which is a string).
The Firebird project has its own OID registered with IANA - 1.3.6.1.4.1.53446 (which in full-name form is
iso.org.dod.internet.private.enterprise.firebird) and maintains its own OID tree on GitHub, see
the firebird-uuid repository. Firebird Butler, Saturnin, solution providers, protocols, etc. all have
their own OID nodes and thus specific UUIDs (and names).
To facilitate work with UUIDs, Saturnin’s own OID registry is provided, which is used to translate between
UUIDs and OIDs, display additional information, etc. Before using this registry, it is necessary to fill
the appropriate records with the update oids command.
Command
update oids.Usage:
update oids URL ╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────╮ │ url URL URL to OID node specification │ │ [default: │ │ https://raw.githubusercontent.com/FirebirdSQL/firebird-uuid/master/root.oid] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
Tip
OID names are available along with UUIDs in command completion once OID registry is updated.
Command
list oidsLists registered OIDs.
Usage:
list oids [OPTIONS] ╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────╮ │ --with-name TEXT List only OIDs with this string in name │ │ --show-oids Should OIDs instead UUIDs │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
Example:
> list oids --with-name protocol Registered OIDs with name containing 'protocol' ╭──────────────────────────────────────────────────┬──────────────────────────────────────╮ │ OID Name │ UUID │ ├──────────────────────────────────────────────────┼──────────────────────────────────────┤ │ firebird.butler.protocol │ 15ab5c16-00e4-5fec-8928-c5d785d66729 │ │ firebird.butler.platform.saturnin.protocol │ 2068f86d-9154-58aa-acb2-b09cca9f9d18 │ │ firebird.butler.platform.saturnin.protocol.dummy │ 4c546e1d-f208-50fb-8b67-100520cb599f │ │ firebird.butler.platform.saturnin.protocol.iccp │ 9b7ac9a3-d684-5955-b0ae-2f69e8666868 │ │ firebird.butler.protocol.dummy │ a86ff2d2-73eb-593f-8b14-f2f7af0233d1 │ │ firebird.butler.protocol.fbsp │ 98bd50a9-5863-551d-b19a-a76e2b2ee4d4 │ │ firebird.butler.protocol.fbdp │ 34209338-6370-5e24-a28a-802814e6327c │ ╰──────────────────────────────────────────────────┴──────────────────────────────────────╯Command
show oidShow information about OID.
Usage:
show oid [OID] ╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────╮ │ oid [OID] OID name or GUID │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
Example:
> show oid firebird.butler.platform.saturnin OID: 1.3.6.1.4.1.53446.1.1.0 UID: 46cd9e8a-c697-5cb5-abb5-bceac5a17075 Node name: saturnin Full name: firebird.butler.platform.saturnin Description: Firebird Butler Development & Deployment Platform in Python 3, also provides reference implementations for Firebird Burtler standards Contact: Pavel Císař E-mail: pcisar@users.sourceforge.net Site: https://firebirdsql.org/en/saturnin/ Node spec.: https://raw.githubusercontent.com/FirebirdSQL/saturnin/master/oid/saturnin.oid Node type: NODE Parent spec.: https://raw.githubusercontent.com/FirebirdSQL/Butler/master/oid/platforms.oid
Working with recipes¶
Recipes describing component configuration (including structural composition) can be used in two ways:
As a required parameter for the
saturnin-serviceandsaturnin-bundletools. This method is particularly suitable for the development and testing of saturnin-based solutions.As executable commands in saturnin console. This method is preferred for routine deployment of already created solutions.
Saturnin console works with its own recipe repository. Recipes can be created directly in the console,
or recipes created in another way can be installed (e.g. from the solution supplier or from the Saturnin
application). All registered recipes can then be activated directly with the run command.
Important
Due to the above, there is a restriction on the uniqueness of the recipe name. However, the same recipe can be installed repeatedly, under different names (e.g. if several different variants of the recipe are needed).
Tip
In addition to multiple installations of similar recipes under different names, different variants can be
created directly in the recipe. This is because a recipe can contain multiple sections describing the running
components. When activating the recipe, you can specify (using the --section option) an alternative name
for the section describing the components used.
The names and structure of the sections differ according to the container used.
By default:
the
saturnin-bundlecontainer uses a “bundle” section with aServiceBundleConfigstructure that refers to the configuration sections of individual components.the
saturnin-servicecontainer uses a “service” section with aServiceExecConfigstructure. The same section is then used to configure the running component.
The saturnin console provides next commands for work with recipes:
Command
create recipe.Creates a recipe template that uses the specified Butler services. Such a template contains only default settings and usually needs to be modified to achieve the desired results.
Note
The newly created recipe is automatically opened in the default editor for necessary modifications. After saving, the recipe is automatically registered under the
runcommand, so it can be run immediately.Usage:
create recipe [OPTIONS] NAME COMPONENTS... ╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────╮ │ * recipe_name NAME Recipe name │ │ [default: None] │ │ [required] │ │ * components COMPONENTS... Recipe components │ │ [default: None] │ │ [required] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯ ╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────╮ │ --plain Create recipe without comments │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
Command
list recipes.Lists installed (registered) Saturnin recipes.
Usage:
list recipes
Example:
> list recipes Installed recipes ╭────────────┬─────────┬────────────────┬─────┬───────────────────────────────────────────────────────╮ │ Name │ Type │ Execution mode │ App │ Description │ ├────────────┼─────────┼────────────────┼─────┼───────────────────────────────────────────────────────┤ │ master │ BUNDLE │ NORMAL │ ✖ │ This is all-in-one bundle recipe with various │ │ │ │ │ │ alternatives. │ │ log-print │ BUNDLE │ NORMAL │ ✖ │ Sample processing pipeline for log from local │ │ │ │ │ │ Firebird server. │ │ dummy │ SERVICE │ DAEMON │ ✖ │ Dummy service for test purposes. │ │ print-file │ BUNDLE │ NORMAL │ ✔ │ Print text file. │ │ test │ BUNDLE │ NORMAL │ ✖ │ Not provided │ ╰────────────┴─────────┴────────────────┴─────┴───────────────────────────────────────────────────────╯Command
show recipe.It analyzes the content of the recipe and displays its structure and configuration according to the default sections of the container configuration. If the recipe contains several variants, it is necessary to enter the name of the specific section for the configuration of the container to display them.
Alternatively, it is possible to display the entire recipe in text form (with syntax highlighting).
Usage:
show recipe [OPTIONS] RECIPE_NAME ╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────╮ │ * recipe_name TEXT Recipe name │ │ [default: None] │ │ [required] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯ ╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────╮ │ --section TEXT Configuration section name │ │ [default: None] │ │ --raw Print recipe file content instead normal output │ │ --help Show this message and exit. │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
Example:
> show recipe log-print Name: log-print Type: BUNDLE Exec. mode: NORMAL Executor: DEFAULT Application: Description: Sample processing pipeline for log from local Firebird server. Recipe components (default section) ╭─────────────┬───────────────────────────────┬─────────┬─────────────────────────────────────────────╮ │ Cfg. name │ Component │ Version │ Description │ ├─────────────┼───────────────────────────────┼─────────┼─────────────────────────────────────────────┤ │ from-server │ saturnin.firebird.log.fromsrv │ 0.2.1 │ Firebird log from server provider │ │ │ │ │ microservice │ │ log-parser │ saturnin.firebird.log.parser │ 0.2.1 │ Firebird log parser microservice │ │ log-print │ saturnin.proto.printer │ 0.2.1 │ Protobuf data printer microservice │ │ writer │ saturnin.text.writer │ 0.2.1 │ Text writer microservice │ ╰─────────────┴───────────────────────────────┴─────────┴─────────────────────────────────────────────╯ from-server configuration ╭─────────────────────────┬──────────────────────────────────╮ │ Parameter │ Value │ ├─────────────────────────┼──────────────────────────────────┤ │ agent │ 212657dc26185f4ba8f5d8d42e99fe7e │ │ logging_id │ None │ │ stop_on_close │ True │ │ pipe │ pipe-1 │ │ pipe_address │ inproc://pipe-1 │ │ pipe_mode │ BIND │ │ pipe_format │ text/plain;charset=utf-8 │ │ batch_size │ 50 │ │ ready_schedule_interval │ 1000 │ │ server │ local │ │ max_chars │ 65535 │ ╰─────────────────────────┴──────────────────────────────────╯ log-parser configuration ╭────────────────────────────────┬───────────────────────────────────────────────────────────────────╮ │ Parameter │ Value │ ├────────────────────────────────┼───────────────────────────────────────────────────────────────────┤ │ agent │ a93975c3d8c45e19b89809391cafa8d8 │ │ logging_id │ None │ │ propagate_input_error │ True │ │ input_pipe │ pipe-1 │ │ input_pipe_address │ inproc://pipe-1 │ │ input_pipe_mode │ CONNECT │ │ input_pipe_format │ text/plain;charset=utf-8 │ │ input_batch_size │ 5 │ │ input_ready_schedule_interval │ 1000 │ │ output_pipe │ pipe-2 │ │ output_pipe_address │ inproc://pipe-2 │ │ output_pipe_mode │ BIND │ │ output_pipe_format │ application/x.fb.proto;type=saturnin.core.protobuf.fblog.LogEntry │ │ output_batch_size │ 50 │ │ output_ready_schedule_interval │ 1000 │ ╰────────────────────────────────┴───────────────────────────────────────────────────────────────────╯ log-print configuration ╭────────────────────────────────┬────────────────────────────────────────────────────────────────────╮ │ Parameter │ Value │ ├────────────────────────────────┼────────────────────────────────────────────────────────────────────┤ │ agent │ a58a9b30117a529e8084b9f8daf96d3e │ │ logging_id │ None │ │ propagate_input_error │ True │ │ input_pipe │ pipe-2 │ │ input_pipe_address │ inproc://pipe-2 │ │ input_pipe_mode │ CONNECT │ │ input_pipe_format │ application/x.fb.proto;type=saturnin.core.protobuf.fblog.LogEntry │ │ input_batch_size │ 5 │ │ input_ready_schedule_interval │ 1000 │ │ output_pipe │ pipe-3 │ │ output_pipe_address │ inproc://pipe-3 │ │ output_pipe_mode │ BIND │ │ output_pipe_format │ text/plain;charset=utf-8 │ │ output_batch_size │ 5 │ │ output_ready_schedule_interval │ 1000 │ │ template │ {data.timestamp.ToDatetime()!s} │ │ │ {utils.short_enum_name('saturnin.core.protobuf.SeverityLevel',dat… │ │ │ {utils.short_enum_name('saturnin.core.protobuf.fblog.FirebirdFaci… │ │ │ data.facility):10} {data.code} {data.message}{utils.LF} │ │ func │ None │ ╰────────────────────────────────┴────────────────────────────────────────────────────────────────────╯ writer configuration ╭─────────────────────────┬──────────────────────────────────╮ │ Parameter │ Value │ ├─────────────────────────┼──────────────────────────────────┤ │ agent │ 4e606fdf3fa95d18a7149448a8085aab │ │ logging_id │ None │ │ stop_on_close │ True │ │ pipe │ pipe-3 │ │ pipe_address │ inproc://pipe-3 │ │ pipe_mode │ CONNECT │ │ pipe_format │ text/plain;charset=utf-8 │ │ batch_size │ 5 │ │ ready_schedule_interval │ 1000 │ │ filename │ stdout │ │ file_format │ text/plain;charset=utf-8 │ │ file_mode │ WRITE │ ╰─────────────────────────┴──────────────────────────────────╯Example:
> show recipe --raw log-print [saturnin.recipe] recipe_type = bundle execution_mode = normal description = Sample processing pipeline for log from local Firebird server. [bundle] agents = from-server, log-parser, log-print, writer [from-server] agent = 212657dc-2618-5f4b-a8f5-d8d42e99fe7e pipe = pipe-1 pipe_address = inproc://${pipe} pipe_mode = bind pipe_format = text/plain;charset=utf-8 server = local [log-parser] agent = a93975c3-d8c4-5e19-b898-09391cafa8d8 ; Filter config input_pipe = pipe-1 input_pipe_address = inproc://${input_pipe} input_pipe_mode = connect input_pipe_format = text/plain;charset=utf-8 input_batch_size = 5 ; output_pipe = pipe-2 output_pipe_address = inproc://${output_pipe} output_pipe_mode = bind output_batch_size = 50 [log-print] agent = a58a9b30-117a-529e-8084-b9f8daf96d3e ; Filter config input_pipe = pipe-2 input_pipe_address = inproc://${input_pipe} input_pipe_mode = connect input_pipe_format = application/x.fb.proto;type=saturnin.core.protobuf.fblog.LogEntry input_batch_size = 5 ; output_pipe = pipe-3 output_pipe_address = inproc://${output_pipe} output_pipe_mode = bind output_batch_size = 5 ; template = {data.timestamp.ToDatetime()!s} {utils.short_enum_name('saturnin.core.protobuf.SeverityLevel',data.level):8} {utils.short_enum_name('saturnin.core.protobuf.fblog.FirebirdFacility', data.facility):10} {data.code} {data.message}{utils.LF} [writer] agent = 4e606fdf-3fa9-5d18-a714-9448a8085aab pipe = pipe-3 pipe_address = inproc://${pipe} pipe_mode = connect pipe_format = text/plain;charset=utf-8 ;filename = /home/job/python/data/parsed.log filename = stdout file_format = text/plain;charset=utf-8 file_mode = write
Command
edit recipe.Edit recipe.
Usage:
edit recipe RECIPE_NAME ╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────╮ │ * recipe_name TEXT Recipe name │ │ [default: None] │ │ [required] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
Command
install recipe.Installs a new recipe from an external recipe file or from an installed application. Once installed, recipe can be executed immediately with the
runcommand.Note
It performs only basic recipe validation, i.e. that required sections (recipe + container) are present, and that components required by recipe are installed.
Usage:
install recipe [OPTIONS] ╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────╮ │ --recipe-name TEXT Recipe name (default is recipe file name / application name) │ │ [default: None] │ │ --recipe-file FILE Recipe file. Mutually exclusive with --app-id │ │ [default: None] │ │ --app-id TEXT Application UID or name │ │ [default: None] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
Command
uninstall recipe.Uninstall recipe. Can optionally save the recipe to file before it’s deleted.
Usage:
uninstall recipe [OPTIONS] [RECIPE_NAME] ╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────╮ │ recipe_name [RECIPE_NAME] The name of the recipe to be uninstalled │ │ [default: None] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯ ╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────╮ │ --save-to FILE File where recipe should be saved before it's removed │ │ [default: None] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
Command
run.The “run” command is used to activate the containers according to the installed recipes. Internally, each installed recipe is made available as a separate sub-command under the “run” command. The parameters of the individual commands depend on the container or application used in the recipe.
The specific behavior of the command depends on the components used, the application (if used) and the execution mode:
Recipe commands with
DAEMONexecution mode start the container as a separate daemon process, and the console immediately returns to the command prompt.Recipe commands with
NORMALexecution mode will start the container, and the console will return to the command prompt only after the container has finished running.
Daemon processes¶
Containers can be run as daemon processes using the saturnin-daemon tool, or using the saturnin console and
installed recipes with DAEMON execution mode. The management of these daemon processes differs depending on
the activation method:
Processes started with
saturnin-daemonmust be managed with your own resources (see documentation for the saturnin-daemon tool and working with PID files).Processes started using the saturnin console can be managed in the console using commands for working with daemon processes.
Command
list daemons.List running Saturnin daemons.
Usage:
list daemons
Example:
> list daemons Running daemons ╭───────┬──────────┬────────┬──────────────────────────────────╮ │ PID │ Status │ Recipe │ Description │ ├───────┼──────────┼────────┼──────────────────────────────────┤ │ 14082 │ sleeping │ dummy2 │ Dummy service for test purposes. │ │ 14087 │ sleeping │ dummy2 │ Dummy service for test purposes. │ ╰───────┴──────────┴────────┴──────────────────────────────────╯Command
show daemon.Show information about running Saturnin daemon.
Usage:
show daemon PID ╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────╮ │ * pid INTEGER [default: None] [required] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯
Example:
> show daemon 14082 Process 14082 ╭─────────────────┬───────────────────────────────────────────────────────────────────────────────────╮ │ Status: │ sleeping │ │ Created: │ 2023-03-27 15:14:49 │ │ Run time: │ 0:02:58.567710 │ ├─────────────────┼───────────────────────────────────────────────────────────────────────────────────┤ │ # threads: │ 4 │ │ # files: │ 12 │ │ # INET con.: │ 0 │ ├─────────────────┼───────────────────────────────────────────────────────────────────────────────────┤ │ Name: │ saturnin-bundle │ │ Executable: │ /usr/bin/python3.9 │ │ Cmd. line: │ /home/job/python/envs/saturnin/bin/python, │ │ │ /home/job/python/envs/saturnin/bin/saturnin-bundle, -q, │ │ │ /home/job/python/projects/saturnin/home/data/recipes/dummy2.cfg │ │ CWD: │ /home/job/python/projects/saturnin │ │ User: │ pcisar │ ├─────────────────┼───────────────────────────────────────────────────────────────────────────────────┤ │ CPU user: │ 0.28 │ │ CPU system: │ 0.04 │ │ CPU I/O wait: │ 0.0 │ ├─────────────────┼───────────────────────────────────────────────────────────────────────────────────┤ │ RSS (bytes): │ 26.84 MiB │ │ VMS (bytes): │ 205.64 MiB │ ├─────────────────┼───────────────────────────────────────────────────────────────────────────────────┤ │ Read count: │ 1013 │ │ Write count: │ 4 │ │ Bytes read: │ 1445888 │ │ Bytes written: │ 0 │ ╰─────────────────┴───────────────────────────────────────────────────────────────────────────────────╯
Command
stop daemon.Stop running Saturnin daemon.
Usage:
stop daemon PID ╭─ Arguments ─────────────────────────────────────────────────────────────────────────────────────────╮ │ * pid INTEGER [default: None] [required] │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────╯