Spire Trading Inc. - User contributions [en]

Server Overview

2025-12-01T22:40:08Z

Kamal:

= Server Overview =
Spire is a distributed trading platform composed of multiple specialized servers that work together to provide a complete trading infrastructure. Each server is independently deployable, communicates through well-defined service interfaces, and authenticates through a central Service Locator.
== Architecture Principles ==

'''Service-Oriented Design''' - Each server provides a focused set of functionality

'''Distributed Deployment''' - Services can be deployed across multiple machines for scalability

'''Centralized Authentication''' - All services authenticate through the Service Locator

'''Persistent Storage''' - Critical data persisted to MySQL for durability

'''Real-time Updates''' - Services publish updates to subscribed clients

== Service Categories ==
=== Infrastructure Services ===

'''[[Service Locator]]''' - Centralized authentication, account management, and service discovery

'''[[Uid Server]]''' - Generates unique identifiers for orders and other entities

=== Account & Administration ===

'''[[Administration Server]]''' - Manages account roles, entitlements, risk parameters, and trading permissions

'''[[Web Portal]]''' - Web-based interface for account management and administrative tasks

=== Market Data ===

'''[[Definitions Server]]''' - Provides reference data (countries, currencies, venues, trading schedules)

'''[[Market Data Server]]''' - Receives, sequences, and persists market data from feed clients

'''[[Market Data Relay Server]]''' - Load-balanced distribution of market data to end users

'''[[Charting Server]]''' - Processes time series queries for candlestick and technical analysis data

=== Trading Controls ===

'''[[Compliance Server]]''' - Manages compliance rules and monitors violations

'''[[Risk Server]]''' - Monitors profit/loss, enforces risk limits, manages account state transitions

'''[[Order Execution Server]]''' - Routes orders to venues, tracks execution, enforces controls

== Common Operational Patterns ==
All servers follow consistent operational patterns:

Configuration via YAML files

Setup scripts for generating configurations

Standard start/stop/check scripts

Timestamped log files in logs/ directory

PID-based process management

== System-Wide Management ==
In addition to individual server management scripts, the platform provides aggregate scripts for managing all servers collectively.

=== install.sh ===
The <code>install.sh</code> script is designed for the latest Ubuntu Server LTS and performs a complete system installation including:
* Installing system dependencies (build tools, MySQL, Node.js, Python packages)
* Building all server applications
* Configuring MySQL database and creating the spire schema
* Creating service accounts in the Service Locator
* Setting up initial permissions and directory structure
* Configuring all servers with appropriate network settings

Usage:
<pre>
./install.sh -p [password] [options]

Required:
-p Password for Spire services

Optional:
-u MySQL username (default: spireadmin)
-m MySQL password (default: same as -p)
-i Global network interface (default: auto-detected)
</pre>

The installation script automatically creates service accounts for all servers and assigns them to the appropriate permission groups.

=== setup.py ===
The <code>setup.py</code> script configures all servers by propagating common settings across individual server configurations:

<pre>
python setup.py [options]

Optional:
--local Local network interface
--world Global/public network interface
--address Service Locator address
--password Service account password
--mysql_address MySQL server address
--mysql_username MySQL username
--mysql_password MySQL password
--mysql_schema MySQL schema name
</pre>

This script calls each server's individual setup script with appropriate parameters, ensuring consistent configuration across the platform.

=== start.sh ===
The <code>start.sh</code> script starts all servers in dependency order:

# Service Locator
# Uid Server
# Definitions Server
# Administration Server
# Market Data Server
# Market Data Relay Server
# Charting Server
# Compliance Server
# Order Execution Server
# Risk Server
# Web Portal
# Market Data Feed Client

After starting all servers, the script automatically runs <code>reset_risk_states.py</code> to initialize risk state for the trading session.

=== stop.sh ===
The <code>stop.sh</code> script gracefully stops all servers in reverse dependency order, ensuring that dependent services shut down before the services they depend on.

=== check.sh ===
The <code>check.sh</code> script verifies that all servers are running and reports the status of each service. It exits with a non-zero status if any server is not running, making it suitable for monitoring and health check automation.

Server Overview

2025-12-01T22:39:52Z

Kamal: Created page with "= Server Overview = Spire is a distributed trading platform composed of multiple specialized servers that work together to provide a complete trading infrastructure. Each serv..."

= Server Overview =
Spire is a distributed trading platform composed of multiple specialized servers that work together to provide a complete trading infrastructure. Each server is independently deployable, communicates through well-defined service interfaces, and authenticates through a central Service Locator.
== Architecture Principles ==

'''Service-Oriented Design''' - Each server provides a focused set of functionality

'''Distributed Deployment''' - Services can be deployed across multiple machines for scalability

'''Centralized Authentication''' - All services authenticate through the Service Locator

'''Persistent Storage''' - Critical data persisted to MySQL for durability

'''Real-time Updates''' - Services publish updates to subscribed clients

== Service Categories ==
=== Infrastructure Services ===

'''[[Service Locator]]''' - Centralized authentication, account management, and service discovery

'''[[Uid Server]]''' - Generates unique identifiers for orders and other entities

=== Account & Administration ===

'''[[Administration Server]]''' - Manages account roles, entitlements, risk parameters, and trading permissions

'''[[Web Portal]]''' - Web-based interface for account management and administrative tasks

=== Market Data ===

'''[[Definitions Server]]''' - Provides reference data (countries, currencies, venues, trading schedules)

'''[[Market Data Server]]''' - Receives, sequences, and persists market data from feed clients

'''[[Market Data Relay Server]]''' - Load-balanced distribution of market data to end users

'''[[Charting Server]]''' - Processes time series queries for candlestick and technical analysis data

=== Trading Controls ===

'''[[Compliance Server]]''' - Manages compliance rules and monitors violations

'''[[Risk Server]]''' - Monitors profit/loss, enforces risk limits, manages account state transitions

'''[[Order Execution Server]]''' - Routes orders to venues, tracks execution, enforces controls

== Common Operational Patterns ==
All servers follow consistent operational patterns:

Configuration via YAML files

Setup scripts for generating configurations

Standard start/stop/check scripts

Timestamped log files in logs/ directory

PID-based process management

== System-Wide Management ==
In addition to individual server management scripts, the platform provides aggregate scripts for managing all servers collectively.

=== install.sh ===
The <code>install.sh</code> script is designed for Ubuntu Server LTS and performs a complete system installation including:
* Installing system dependencies (build tools, MySQL, Node.js, Python packages)
* Building all server applications
* Configuring MySQL database and creating the spire schema
* Creating service accounts in the Service Locator
* Setting up initial permissions and directory structure
* Configuring all servers with appropriate network settings

Usage:
<pre>
./install.sh -p [password] [options]

Required:
-p Password for Spire services

Optional:
-u MySQL username (default: spireadmin)
-m MySQL password (default: same as -p)
-i Global network interface (default: auto-detected)
</pre>

The installation script automatically creates service accounts for all servers and assigns them to the appropriate permission groups.

=== setup.py ===
The <code>setup.py</code> script configures all servers by propagating common settings across individual server configurations:

<pre>
python setup.py [options]

Optional:
--local Local network interface
--world Global/public network interface
--address Service Locator address
--password Service account password
--mysql_address MySQL server address
--mysql_username MySQL username
--mysql_password MySQL password
--mysql_schema MySQL schema name
</pre>

This script calls each server's individual setup script with appropriate parameters, ensuring consistent configuration across the platform.

=== start.sh ===
The <code>start.sh</code> script starts all servers in dependency order:

# Service Locator
# Uid Server
# Definitions Server
# Administration Server
# Market Data Server
# Market Data Relay Server
# Charting Server
# Compliance Server
# Order Execution Server
# Risk Server
# Web Portal
# Market Data Feed Client

After starting all servers, the script automatically runs <code>reset_risk_states.py</code> to initialize risk state for the trading session.

=== stop.sh ===
The <code>stop.sh</code> script gracefully stops all servers in reverse dependency order, ensuring that dependent services shut down before the services they depend on.

=== check.sh ===
The <code>check.sh</code> script verifies that all servers are running and reports the status of each service. It exits with a non-zero status if any server is not running, making it suitable for monitoring and health check automation.

Web Portal

2025-12-01T22:30:25Z

Kamal: Created page with "= Web Portal = The Web Portal provides web-based access to account management and administrative functions through both an web API and an interactive HTML interface. It serves..."

= Web Portal =
The Web Portal provides web-based access to account management and administrative functions through both an web API and an interactive HTML interface. It serves as the primary user interface for creating and managing trading accounts, configuring market data entitlements, defining compliance rules, setting risk parameters, and monitoring account activity. By aggregating functionality from multiple backend services, the Web Portal enables administrators and managers to perform operational tasks through a unified web application.

The Web Portal integrates with the Service Locator for authentication and communicates with the Administration Server, Compliance Server, Risk Server, and Order Execution Server to provide comprehensive account management capabilities.

== Configuration ==
The Web Portal is configured via a YAML file that defines its network interface and Service Locator integration. Below is the structure of the configuration file with example values:

<pre>
---
server:
# Primary network interface and port the Web Portal binds to.
interface: "0.0.0.0:8080"

# List of addresses the server is reachable at (for registration with Service Locator).
# Typically includes both public-facing and local addresses.
addresses: ["198.51.100.5:8080", "10.0.0.5:8080"]

service_locator:
# The address of the Service Locator (host:port).
address: "10.0.0.5:20000"

# The account username used by the Web Portal to authenticate with the Service Locator.
username: web_portal_service

# The password for the Web Portal's Service Locator account.
password: admin_password
...
</pre>

== Functionality ==
The Web Portal provides access to the following administrative and operational capabilities:

=== Account Management ===
* Create new trading accounts
* Assign accounts to groups and directories
* Configure account permissions and access controls
* Set account roles (trader, manager, administrator, service)
* Modify account credentials and settings

=== Market Data Entitlements ===
* View available market data packages and pricing
* Assign entitlements to accounts or groups
* Configure per-venue and per-message-type permissions

=== Compliance Configuration ===
* Define compliance rules and rule schemas
* Assign rules to accounts, groups, or directories
* Configure rule states (active, passive, disabled)
* Monitor compliance rule violations
* Review historical compliance events

=== Risk Management ===
* Configure risk parameters per account
* Set buying power limits
* Define net loss thresholds
* Configure risk state transitions
* Monitor real-time profit and loss
* View position summaries and inventory snapshots

=== Order and Execution Monitoring ===
* View order history and execution reports
* Monitor active orders
* Review trade activity by account or region
* Generate reports on trading performance
* Track order routing and fill statistics

=== Administrative Tools ===
* User session management
* System configuration and settings
* Audit logs and activity tracking
* Service health monitoring
* Operational reports and analytics

== Access Control ==
The Web Portal enforces permissions based on account roles:

* '''Administrators''' have full access to all management functions
* '''Managers''' can manage accounts within their assigned groups
* '''Traders''' can view their own account details and activity
* '''Service''' accounts have restricted programmatic access

All operations are authenticated through the Service Locator and logged for audit purposes.

== Installation & Setup ==
A <code>setup.py</code> script is provided to generate the final <code>config.yml</code> from the <code>config.default.yml</code> template.

The script supports the following arguments:

<pre>
python setup.py
--local 0.0.0.0 # Local interface (default: auto-detected IP)
--world 198.51.100.5 # Global/public interface (optional)
--address 10.0.0.5:20000 # Service Locator address (default: local_interface:20000)
--password [REQUIRED] # Service password for authentication
</pre>

== Operations ==
The Web Portal is controlled using three operational scripts: <code>start.sh</code>, <code>stop.sh</code>, and <code>check.sh</code>.

Log files are generated in the format:
<pre>
srv_YYYYMMDD_HH_MM_SS.log
</pre>

Upon startup, older log files are moved into the <code>logs/</code> directory.

=== check.sh ===
The <code>check.sh</code> script verifies whether the server is currently running by inspecting the PID recorded in <code>pid.lock</code> and testing whether the associated process exists.

If the server is not running, it prints:
<pre>
WebPortal is not running.
</pre>

=== start.sh ===
The <code>start.sh</code> script:
* Exits immediately if the server is already running
* Creates a <code>logs/</code> directory if necessary
* Moves any existing <code>srv_*.log</code> files into <code>logs/</code>
* Starts the <code>WebPortal</code> process in the background
* Writes the PID to <code>pid.lock</code>
* Reads network interfaces from <code>config.yml</code> and waits until the server is listening on at least one configured address

This ensures the server is fully initialized before the script exits.

=== stop.sh ===
The <code>stop.sh</code> script:
* Reads the PID from <code>pid.lock</code>
* Sends <code>SIGINT</code> to request a graceful shutdown
* Waits for termination using exponential backoff (up to 300 seconds)
* Sends <code>SIGKILL</code> if the server fails to stop cleanly
* Appends a forced-termination message to the most recent log file (if applicable)
* Removes the <code>pid.lock</code> file

This guarantees consistent shutdown behavior across normal and exceptional conditions.

Order Execution Server

2025-12-01T22:25:42Z

Kamal:

= Order Execution Server =
The Order Execution Server receives order submissions from clients, routes them to appropriate execution venues or brokers, and tracks the lifecycle of each order from submission through completion. It publishes execution reports to subscribed clients, enforces compliance rules and risk parameters, and maintains a persistent record of all order activity. By centralizing order routing and monitoring, the server ensures consistent application of trading controls while providing real-time visibility into order status across the system.

The Order Execution Server integrates with the Service Locator for authentication, the Compliance Server for rule validation, the Risk Server for position and loss monitoring, and a MySQL database to persist order records and execution reports.

== Configuration ==
The Order Execution Server is configured via a YAML file that defines network interfaces, database connection, and Service Locator integration. Below is the structure of the configuration file with example values:

<pre>
---
server:
# Primary network interface and port the Order Execution Server binds to.
interface: "0.0.0.0:20700"

# List of addresses the server is reachable at (for registration with Service Locator).
# Typically includes both public-facing and local addresses.
addresses: ["198.51.100.5:20700", "10.0.0.5:20700"]

data_store:
# The address of the MySQL server (host:port).
address: "127.0.0.1:3306"

# The username for authenticating with MySQL.
username: spireadmin

# The password for the MySQL user.
password: 1234

# The name of the database schema where data is stored.
schema: spire

service_locator:
# The address of the Service Locator (host:port).
address: "10.0.0.5:20000"

# The account username used by the Order Execution Server to authenticate with the Service Locator.
username: order_execution_service

# The password for the Order Execution Server's Service Locator account.
password: admin_password
...
</pre>

== Functionality ==

=== Order Routing ===
The server routes orders to their specified destinations based on:

* Destination configuration (venue, broker, or internal order type like MOE)
* Account permissions and entitlements
* Compliance rule validation
* Risk parameter enforcement

Orders that violate compliance rules or risk limits are rejected before routing.

=== Order Monitoring ===
Clients can subscribe to receive execution reports for:

* Orders submitted by a specific account
* Individual orders by order ID

Subscriptions enable real-time tracking of order activity across portfolios or trading desks.

=== Compliance and Risk Enforcement ===
Before accepting an order, the server validates:

* Compliance rules defined for the account or group
* Risk parameters including buying power and loss limits
* Account state (active, closed orders, disabled)
* Trading permissions and entitlements

Orders that fail validation are immediately rejected with an appropriate error message.

=== Order Cancellation ===
Clients can request order cancellation. The server:

# Validates cancel permission
# Routes cancel request to the destination
# Updates order state based on destination response
# Publishes final execution report

== Utility Scripts ==

=== cancel_order.py ===
A utility for bulk order cancellation supporting multiple use cases:

'''Cancel by order ID:'''
<pre>
python cancel_order.py
--config config.yml
--order 12345
--message "Manual cancellation"
</pre>

'''Cancel by account and region:'''
<pre>
python cancel_order.py
--config config.yml
--account trader1
--region CA # Country code, venue, or security
--begin "2024-01-01"
--end "2024-12-31"
--message "End of day cancel"
</pre>

'''Cancel by region (all accounts):'''
<pre>
python cancel_order.py
--config config.yml
--region XTSE # Venue code
--begin "2024-01-01"
--end "2024-12-31"
--connections 8 # Parallel connections for performance
--message "Market close"
</pre>

The script supports parallel processing using multiple service connections to efficiently cancel large numbers of orders.

== Installation & Setup ==
A <code>setup.py</code> script is provided to generate the final <code>config.yml</code> from the <code>config.default.yml</code> template.

The script supports the following arguments:

<pre>
python setup.py
--local 0.0.0.0 # Local interface (default: auto-detected IP)
--world 198.51.100.5 # Global/public interface (optional)
--address 10.0.0.5:20000 # Service Locator address (default: local_interface:20000)
--password [REQUIRED] # Service password for authentication
--mysql_address 127.0.0.1:3306 # MySQL server address
--mysql_username spireadmin # MySQL username
--mysql_password secretpw # MySQL password (default: --password if omitted)
--mysql_schema spire # MySQL schema
</pre>

== Operations ==
The Order Execution Server is controlled using three operational scripts: <code>start.sh</code>, <code>stop.sh</code>, and <code>check.sh</code>.

Log files are generated in the format:
<pre>
srv_YYYYMMDD_HH_MM_SS.log
</pre>

Upon startup, older log files are moved into the <code>logs/</code> directory.

=== check.sh ===
The <code>check.sh</code> script verifies whether the server is currently running by inspecting the PID recorded in <code>pid.lock</code> and testing whether the associated process exists.

If the server is not running, it prints:
<pre>
OrderExecutionServer is not running.
</pre>

=== start.sh ===
The <code>start.sh</code> script:
* Exits immediately if the server is already running
* Creates a <code>logs/</code> directory if necessary
* Moves any existing <code>srv_*.log</code> files into <code>logs/</code>
* Starts the <code>OrderExecutionServer</code> process in the background
* Writes the PID to <code>pid.lock</code>
* Reads network interfaces from <code>config.yml</code> and waits until the server is listening on at least one configured address

This ensures the server is fully initialized before the script exits.

=== stop.sh ===
The <code>stop.sh</code> script:
* Reads the PID from <code>pid.lock</code>
* Sends <code>SIGINT</code> to request a graceful shutdown
* Waits for termination using exponential backoff (up to 300 seconds)
* Sends <code>SIGKILL</code> if the server fails to stop cleanly
* Appends a forced-termination message to the most recent log file (if applicable)
* Removes the <code>pid.lock</code> file

This guarantees consistent shutdown behavior across normal and exceptional conditions.

Order Execution Server

2025-12-01T22:25:16Z

Kamal: Created page with "= Order Execution Server = The Order Execution Server receives order submissions from clients, routes them to appropriate execution venues or brokers, and tracks the lifecycle..."

= Order Execution Server =
The Order Execution Server receives order submissions from clients, routes them to appropriate execution venues or brokers, and tracks the lifecycle of each order from submission through completion. It publishes execution reports to subscribed clients, enforces compliance rules and risk parameters, and maintains a persistent record of all order activity. By centralizing order routing and monitoring, the server ensures consistent application of trading controls while providing real-time visibility into order status across the system.

The Order Execution Server integrates with the Service Locator for authentication, the Compliance Server for rule validation, the Risk Server for position and loss monitoring, and a MySQL database to persist order records and execution reports.

== Configuration ==
The Order Execution Server is configured via a YAML file that defines network interfaces, database connection, and Service Locator integration. Below is the structure of the configuration file with example values:

<pre>
---
server:
# Primary network interface and port the Order Execution Server binds to.
interface: "0.0.0.0:20700"

# List of addresses the server is reachable at (for registration with Service Locator).
# Typically includes both public-facing and local addresses.
addresses: ["198.51.100.5:20700", "10.0.0.5:20700"]

data_store:
# The address of the MySQL server (host:port).
address: "127.0.0.1:3306"

# The username for authenticating with MySQL.
username: spireadmin

# The password for the MySQL user.
password: 1234

# The name of the database schema where data is stored.
schema: spire

service_locator:
# The address of the Service Locator (host:port).
address: "10.0.0.5:20000"

# The account username used by the Order Execution Server to authenticate with the Service Locator.
username: order_execution_service

# The password for the Order Execution Server's Service Locator account.
password: admin_password
...
</pre>

== Functionality ==

=== Order Routing ===
The server routes orders to their specified destinations based on:

* Destination configuration (venue, broker, or internal order type like MOE)
* Account permissions and entitlements
* Compliance rule validation
* Risk parameter enforcement

Orders that violate compliance rules or risk limits are rejected before routing.

=== Order Monitoring ===
Clients can subscribe to receive execution reports for:

* Orders submitted by a specific account
* Individual orders by order ID

Subscriptions enable real-time tracking of order activity across portfolios or trading desks.

=== Compliance and Risk Enforcement ===
Before accepting an order, the server validates:

* Compliance rules defined for the account or group
* Risk parameters including buying power and loss limits
* Account state (active, closed orders, disabled)
* Trading permissions and entitlements

Orders that fail validation are immediately rejected with an appropriate error message.

=== Order Cancellation ===
Clients can request order cancellation. The server:

# Validates cancel permission
# Routes cancel request to the destination
# Updates order state based on destination response
# Publishes final execution report

== Utility Scripts ==

=== cancel_order.py ===
A utility for bulk order cancellation supporting multiple use cases:

'''Cancel by order ID:'''
<pre>
python cancel_order.py
--config config.yml
--order 12345
--message "Manual cancellation"
</pre>

'''Cancel by account and region:'''
<pre>
python cancel_order.py
--config config.yml
--account trader1
--region CA # Country code, venue, or security
--begin "2024-01-01"
--end "2024-12-31"
--message "End of day cancel"
</pre>

'''Cancel by region (all accounts):'''
<pre>
python cancel_order.py
--config config.yml
--region XTSE # Venue code
--begin "2024-01-01"
--end "2024-12-31"
--connections 8 # Parallel connections for performance
--message "Market close"
</pre>

The script supports parallel processing using multiple service connections to efficiently cancel large numbers of orders.

== Installation & Setup ==
A <code>setup.py</code> script is provided to generate the final <code>config.yml</code> from the <code>config.default.yml</code> template.

The script supports the following arguments:

<pre>
python setup.py
--local 0.0.0.0 # Local interface (default: auto-detected IP)
--world 198.51.100.5 # Global/public interface (optional)
--address 10.0.0.5:20000 # Service Locator address (default: local_interface:20000)
--password [REQUIRED] # Service password for authentication
--mysql_address 127.0.0.1:3306 # MySQL server address
--mysql_username spireadmin # MySQL username
--mysql_password secretpw # MySQL password (default: --password if omitted)
--mysql_schema spire # MySQL schema
</pre>

== Operations ==
The Order Execution Server is controlled using three operational scripts: <code>start.sh</code>, <code>stop.sh</code>, and <code>check.sh</code>.

Log files are generated in the format:
<pre>
srv_YYYYMMDD_HH_MM_SS.log
</pre>

Upon startup, older log files are moved into the <code>logs/</code> directory.

=== check.sh ===
The <code>check.sh</code> script verifies whether the server is currently running by inspecting the PID recorded in <code>pid.lock</code> and testing whether the associated process exists.

If the server is not running, it prints:
<pre>
SimulationOrderExecutionServer is not running.
</pre>

=== start.sh ===
The <code>start.sh</code> script:
* Exits immediately if the server is already running
* Creates a <code>logs/</code> directory if necessary
* Moves any existing <code>srv_*.log</code> files into <code>logs/</code>
* Starts the <code>SimulationOrderExecutionServer</code> process in the background
* Writes the PID to <code>pid.lock</code>
* Reads network interfaces from <code>config.yml</code> and waits until the server is listening on at least one configured address

This ensures the server is fully initialized before the script exits.

=== stop.sh ===
The <code>stop.sh</code> script:
* Reads the PID from <code>pid.lock</code>
* Sends <code>SIGINT</code> to request a graceful shutdown
* Waits for termination using exponential backoff (up to 300 seconds)
* Sends <code>SIGKILL</code> if the server fails to stop cleanly
* Appends a forced-termination message to the most recent log file (if applicable)
* Removes the <code>pid.lock</code> file

This guarantees consistent shutdown behavior across normal and exceptional conditions.

Risk Server

2025-12-01T22:18:57Z

Kamal: Created page with "= Risk Server = The Risk Server monitors account profit and loss, enforces risk limits, and manages account state transitions based on trading activity. It tracks each account..."

= Risk Server =
The Risk Server monitors account profit and loss, enforces risk limits, and manages account state transitions based on trading activity. It tracks each account's net position, buying power consumption, and realized/unrealized profit and loss. When an account exceeds its configured loss threshold, the server automatically transitions the account through risk states—from active trading to closed orders mode, and ultimately to disabled or liquidation if losses continue. By centralizing risk monitoring and enforcement, the server protects both individual accounts and the overall system from excessive exposure.

The Risk Server integrates with the Service Locator for authentication, the Order Execution service to monitor trades, and a MySQL database to persist risk parameters and inventory snapshots.

== Configuration ==
The Risk Server is configured via a YAML file that defines network interfaces, database connection, and Service Locator integration. Below is the structure of the configuration file with example values:

<pre>
---
server:
# Primary network interface and port the Risk Server binds to.
interface: "0.0.0.0:20600"

# List of addresses the server is reachable at (for registration with Service Locator).
# Typically includes both public-facing and local addresses.
addresses: ["198.51.100.5:20600", "10.0.0.5:20600"]

data_store:
# The address of the MySQL server (host:port).
address: "127.0.0.1:3306"

# The username for authenticating with MySQL.
username: spireadmin

# The password for the MySQL user.
password: 1234

# The name of the database schema where data is stored.
schema: spire

service_locator:
# The address of the Service Locator (host:port).
address: "10.0.0.5:20000"

# The account username used by the Risk Server to authenticate with the Service Locator.
username: risk_service

# The password for the Risk Server's Service Locator account.
password: admin_password
...
</pre>

== Risk Parameters ==
Each account is assigned risk parameters that govern its trading limits and risk enforcement behavior:

* '''Currency''' - The currency used for risk calculations and loss limits
* '''Buying Power''' - Maximum amount of capital available for trading
* '''Allowed State''' - The risk state the account is permitted to operate in
* '''Net Loss Threshold''' - Maximum net loss before transitioning to closed orders mode
* '''Transition Time''' - Duration allowed in closed orders mode before automatic liquidation

=== Risk States ===
Accounts progress through the following risk states based on their profit and loss:

* '''Active''' - Normal trading with full permissions
* '''Closed Orders''' - No new orders allowed; existing positions may be reduced
* '''Disabled''' - Trading prohibited; positions must be manually managed
* '''Liquidation''' - Account positions are automatically liquidated

== Functionality ==

=== Real-Time Risk Monitoring ===
The Risk Server continuously tracks:

* Net position by security and currency
* Realized and unrealized profit and loss
* Buying power consumption
* Order and execution activity

When an account's net loss exceeds its configured threshold, the server automatically transitions the account to closed orders mode and starts a timer. If the account does not return to profitability within the transition time, the server escalates to disabled or liquidation state.

=== Inventory Management ===
The server maintains real-time inventory snapshots for each account, tracking:

* Open positions by security
* Average cost basis
* Current market value
* Unrealized profit and loss

Inventory updates occur on every execution report, ensuring accurate position tracking across all trading venues.

=== Regional Risk Control ===
Risk parameters and state transitions can be scoped by region (country, venue, or security). This enables fine-grained control over risk exposure in specific markets or asset classes.

== Utility Scripts ==

=== positions.py ===
Reports current positions for accounts in CSV format:

<pre>
python positions.py
--config config.yml # Configuration file
--account trader1 # Specific account (optional; reports all if omitted)
</pre>

Output format:
<pre>
Account,Security,Currency,Side,Open Quantity,Cost Basis
trader1,AAPL.NASDAQ,USD,Long,100,15000.00
trader1,TSLA.NASDAQ,USD,Short,50,12500.00
</pre>

=== moe.py ===
Manual Order Entry (MOE) utility for opening or closing positions without market execution. These orders are synthetic and do not interact with external venues:

<pre>
python moe.py
--config config.yml # Configuration file
--positions positions.csv # CSV file with positions to MOE
--account trader1 # Specific account (optional)
--region CA # Region filter (country/venue/security)
--open # Open positions
--close # Close positions
</pre>

The positions CSV file must have the format:
<pre>
Account,Security,Currency,Side,Open Quantity,Cost Basis
</pre>

Use cases:
* Opening initial positions for testing
* Closing positions during liquidation
* Adjusting positions after system recovery

=== reset.py ===
Resets risk state for a specific region, clearing accumulated profit/loss tracking:

<pre>
python reset.py
--config config.yml # Configuration file
--region CA # Region to reset (country code, venue, or security)
</pre>

This utility is typically used at the start of a trading day or after maintenance periods to reset risk calculations.

== Installation & Setup ==
A <code>setup.py</code> script is provided to generate the final <code>config.yml</code> from the <code>config.default.yml</code> template.

The script supports the following arguments:

<pre>
python setup.py
--local 0.0.0.0 # Local interface (default: auto-detected IP)
--world 198.51.100.5 # Global/public interface (optional)
--address 10.0.0.5:20000 # Service Locator address (default: local_interface:20000)
--password [REQUIRED] # Service password for authentication
--mysql_address 127.0.0.1:3306 # MySQL server address
--mysql_username spireadmin # MySQL username
--mysql_password secretpw # MySQL password (default: --password if omitted)
--mysql_schema spire # MySQL schema
</pre>

== Operations ==
The Risk Server is controlled using three operational scripts: <code>start.sh</code>, <code>stop.sh</code>, and <code>check.sh</code>.

Log files are generated in the format:
<pre>
srv_YYYYMMDD_HH_MM_SS.log
</pre>

Upon startup, older log files are moved into the <code>logs/</code> directory.

=== check.sh ===
The <code>check.sh</code> script verifies whether the server is currently running by inspecting the PID recorded in <code>pid.lock</code> and testing whether the associated process exists.

If the server is not running, it prints:
<pre>
RiskServer is not running.
</pre>

=== start.sh ===
The <code>start.sh</code> script:
* Exits immediately if the server is already running
* Creates a <code>logs/</code> directory if necessary
* Moves any existing <code>srv_*.log</code> files into <code>logs/</code>
* Starts the <code>RiskServer</code> process in the background
* Writes the PID to <code>pid.lock</code>
* Reads network interfaces from <code>config.yml</code> and waits until the server is listening on at least one configured address

This ensures the server is fully initialized before the script exits.

=== stop.sh ===
The <code>stop.sh</code> script:
* Reads the PID from <code>pid.lock</code>
* Sends <code>SIGINT</code> to request a graceful shutdown
* Waits for termination using exponential backoff (up to 300 seconds)
* Sends <code>SIGKILL</code> if the server fails to stop cleanly
* Appends a forced-termination message to the most recent log file (if applicable)
* Removes the <code>pid.lock</code> file

This guarantees consistent shutdown behavior across normal and exceptional conditions.

Market Data Relay Server

2025-12-01T22:13:09Z

Kamal: Created page with "= Market Data Relay Server = The Market Data Relay Server provides load-balanced distribution of market data to end users. It acts as an intermediary layer between the Market..."

= Market Data Relay Server =
The Market Data Relay Server provides load-balanced distribution of market data to end users. It acts as an intermediary layer between the Market Data Server and client applications, receiving market data updates from the Market Data Server and relaying them to subscribed clients. By deploying multiple relay servers in parallel, the system can distribute client connections across instances, ensuring scalable and resilient market data delivery.

The Market Data Relay Server integrates with the Service Locator for authentication and service registration. Multiple relay servers can operate concurrently to distribute load across the infrastructure.

== Configuration ==
The Market Data Relay Server is configured via a YAML file that defines its network interface and Service Locator integration. Below is the structure of the configuration file with example values:

<pre>
---
server:
# Primary network interface and port the Market Data Relay Server binds to.
interface: "0.0.0.0:22300"

# List of addresses the server is reachable at (for registration with Service Locator).
# Typically includes both public-facing and local addresses.
addresses: ["198.51.100.5:22300", "10.0.0.5:22300"]

service_locator:
# The address of the Service Locator (host:port).
address: "10.0.0.5:20000"

# The account username used by the Market Data Relay Server to authenticate with the Service Locator.
username: market_data_relay_service

# The password for the Market Data Relay Server's Service Locator account.
password: admin_password
...
</pre>

== Architecture ==
The market data distribution architecture consists of three tiers:

# '''Feed Clients''' publish raw market data to the Market Data Server
# '''Market Data Server''' organizes, sequences, and persists market data
# '''Market Data Relay Servers''' distribute market data to end-user clients

This architecture separates data processing from distribution, allowing the Market Data Server to focus on sequencing and storage while relay servers handle the potentially large number of client connections. Multiple relay servers can be deployed to distribute client load geographically or by capacity requirements.

== Installation & Setup ==
A <code>setup.py</code> script is provided to generate the final <code>config.yml</code> from the <code>config.default.yml</code> template.

The script supports the following arguments:

<pre>
python setup.py
--local 0.0.0.0 # Local interface (default: auto-detected IP)
--world 198.51.100.5 # Global/public interface (optional)
--address 10.0.0.5:20000 # Service Locator address (default: local_interface:20000)
--password [REQUIRED] # Service password for authentication
</pre>

== Operations ==
The Market Data Relay Server is controlled using three operational scripts: <code>start.sh</code>, <code>stop.sh</code>, and <code>check.sh</code>.

Log files are generated in the format:
<pre>
srv_YYYYMMDD_HH_MM_SS.log
</pre>

Upon startup, older log files are moved into the <code>logs/</code> directory.

=== check.sh ===
The <code>check.sh</code> script verifies whether the server is currently running by inspecting the PID recorded in <code>pid.lock</code> and testing whether the associated process exists.

If the server is not running, it prints:
<pre>
MarketDataRelayServer is not running.
</pre>

=== start.sh ===
The <code>start.sh</code> script:
* Exits immediately if the server is already running
* Creates a <code>logs/</code> directory if necessary
* Moves any existing <code>srv_*.log</code> files into <code>logs/</code>
* Starts the <code>MarketDataRelayServer</code> process in the background
* Writes the PID to <code>pid.lock</code>
* Reads network interfaces from <code>config.yml</code> and waits until the server is listening on at least one configured address

This ensures the server is fully initialized before the script exits.

=== stop.sh ===
The <code>stop.sh</code> script:
* Reads the PID from <code>pid.lock</code>
* Sends <code>SIGINT</code> to request a graceful shutdown
* Waits for termination using exponential backoff (up to 300 seconds)
* Sends <code>SIGKILL</code> if the server fails to stop cleanly
* Appends a forced-termination message to the most recent log file (if applicable)
* Removes the <code>pid.lock</code> file

This guarantees consistent shutdown behavior across normal and exceptional conditions.

Market Data Server

2025-12-01T22:09:49Z

Kamal: Created page with "= Market Data Server = The Market Data Server receives market data from feed clients, organizes and persists the data, and provides query and subscription services to clients...."

= Market Data Server =
The Market Data Server receives market data from feed clients, organizes and persists the data, and provides query and subscription services to clients. It maintains real-time market data snapshots for each security, tracks security technicals (open, high, low, close, volume), and sequences all market data updates to ensure consistent ordering across the system. The server supports multiple market data types including best bid/offer quotes, book quotes, time and sales, and order imbalances.

The Market Data Server integrates with the Service Locator for authentication and a MySQL database for historical data storage. It operates as two distinct services: a registry server for managing security metadata and subscriptions, and a feed server for receiving and distributing market data updates.

== Configuration ==
The Market Data Server is configured via a YAML file that defines network interfaces for both services, database connection, and Service Locator integration. Below is the structure of the configuration file with example values:

<pre>
---
registry_server:
# Primary network interface and port the Market Data Registry Server binds to.
interface: "0.0.0.0:20300"

# List of addresses the registry server is reachable at (for registration with Service Locator).
addresses: ["198.51.100.5:20300", "10.0.0.5:20300"]

feed_server:
# Primary network interface and port the Market Data Feed Server binds to.
interface: "0.0.0.0:20400"

# List of addresses the feed server is reachable at (for registration with Service Locator).
addresses: ["198.51.100.5:20400", "10.0.0.5:20400"]

data_store:
# The address of the MySQL server (host:port).
address: "127.0.0.1:3306"

# The username for authenticating with MySQL.
username: spireadmin

# The password for the MySQL user.
password: 1234

# The name of the database schema where data is stored.
schema: spire

# Countries for which market data is supported.
countries:
- AU
- CA
- HK
- JP
- US

service_locator:
# The address of the Service Locator (host:port).
address: "10.0.0.5:20000"

# The account username used by the Market Data Server to authenticate with the Service Locator.
username: market_data_service

# The password for the Market Data Server's Service Locator account.
password: admin_password
...
</pre>

== Functionality ==

=== Market Data Types ===
The server processes and distributes the following market data types:

* '''BBO Quotes''' (Best Bid/Offer) - Top-of-book bid and ask prices with sizes
* '''Book Quotes''' - Full order book depth with multiple price levels per side
* '''Time and Sales''' - Executed trade prices, sizes, and timestamps
* '''Order Imbalances''' - Pre-opening and closing auction imbalance data

=== Security Tracking ===
For each security, the server maintains:

* Real-time market data snapshots combining BBO, book depth, and last trade
* Security technicals including open, high, low, close, and volume
* Sequence numbers ensuring consistent ordering of market data updates
* Source tracking to identify which feed client published each data point

=== Data Sequencing ===
All market data updates are assigned monotonically increasing sequence numbers per security and data type. This ensures clients can detect gaps, maintain consistent ordering, and synchronize state reliably.

=== Historical Data Storage ===
Market data is persisted to MySQL with indexed queries supporting:

* Time range queries by security or venue
* Snapshot queries for initial state loading
* Efficient storage of high-frequency data streams

== Utility Scripts ==

=== add_security_info.py ===
A utility script to add or update security metadata in the system:

<pre>
python add_security_info.py
--config config.yml # Configuration file
--symbol AAPL.NASDAQ # Ticker symbol
--name "Apple Inc." # Display name
--sector Technology # Sector (optional)
--board_lot 100 # Board lot size
</pre>

=== dump_data_store.py ===
A backup and restore utility supporting bidirectional data transfer between MySQL and SQLite:

<pre>
python dump_data_store.py
--config config.yml # Configuration file
--start "2024-01-01 00:00:00" # Start time range
--end "2024-12-31 23:59:59" # End time range
--output backup.db # Export MySQL to SQLite
--cores 8 # Number of parallel workers
</pre>

Or to restore from SQLite to MySQL:

<pre>
python dump_data_store.py
--config config.yml
--start "2024-01-01 00:00:00"
--end "2024-12-31 23:59:59"
--input backup.db
--cores 8
</pre>

The script uses multiprocessing to efficiently backup or restore large datasets across multiple securities and venues in parallel.

== Installation & Setup ==
A <code>setup.py</code> script is provided to generate the final <code>config.yml</code> from the <code>config.default.yml</code> template.

The script supports the following arguments:

<pre>
python setup.py
--local 0.0.0.0 # Local interface (default: auto-detected IP)
--world 198.51.100.5 # Global/public interface (optional)
--address 10.0.0.5:20000 # Service Locator address (default: local_interface:20000)
--password [REQUIRED] # Service password for authentication
--mysql_address 127.0.0.1:3306 # MySQL server address
--mysql_username spireadmin # MySQL username
--mysql_password secretpw # MySQL password (default: --password if omitted)
--mysql_schema spire # MySQL schema
</pre>

== Operations ==
The Market Data Server is controlled using three operational scripts: <code>start.sh</code>, <code>stop.sh</code>, and <code>check.sh</code>.

Log files are generated in the format:
<pre>
srv_YYYYMMDD_HH_MM_SS.log
</pre>

Upon startup, older log files are moved into the <code>logs/</code> directory.

=== check.sh ===
The <code>check.sh</code> script verifies whether the server is currently running by inspecting the PID recorded in <code>pid.lock</code> and testing whether the associated process exists.

If the server is not running, it prints:
<pre>
MarketDataServer is not running.
</pre>

=== start.sh ===
The <code>start.sh</code> script:
* Exits immediately if the server is already running
* Creates a <code>logs/</code> directory if necessary
* Moves any existing <code>srv_*.log</code> files into <code>logs/</code>
* Starts the <code>MarketDataServer</code> process in the background
* Writes the PID to <code>pid.lock</code>
* Reads network interfaces from <code>config.yml</code> and waits until the server is listening on at least one configured address

This ensures the server is fully initialized before the script exits.

=== stop.sh ===
The <code>stop.sh</code> script:
* Reads the PID from <code>pid.lock</code>
* Sends <code>SIGINT</code> to request a graceful shutdown
* Waits for termination using exponential backoff (up to 300 seconds)
* Sends <code>SIGKILL</code> if the server fails to stop cleanly
* Appends a forced-termination message to the most recent log file (if applicable)
* Removes the <code>pid.lock</code> file

This guarantees consistent shutdown behavior across normal and exceptional conditions.

Definitions Server

2025-12-01T22:03:44Z

Kamal: Created page with "= Definitions Server = The Definitions Server provides centralized reference data for the trading platform. It disseminates system-wide definitions including country codes, cu..."

= Definitions Server =
The Definitions Server provides centralized reference data for the trading platform. It disseminates system-wide definitions including country codes, currencies, trading venues, order routing destinations, time zones, exchange rates, compliance rule schemas, and trading schedules. By centralizing these definitions, the server ensures consistent metadata across all components of the system.

The Definitions Server integrates with the Service Locator for authentication and service registration. All reference data is loaded from YAML and CSV configuration files at startup.

== Configuration ==
The Definitions Server is configured via a YAML file that defines network interfaces, Service Locator integration, and paths to reference data files. Below is the structure of the configuration file with example values:

<pre>
---
server:
# Primary network interface and port the Definitions Server binds to.
interface: "0.0.0.0:20200"

# List of addresses the server is reachable at (for registration with Service Locator).
# Typically includes both public-facing and local addresses.
addresses: ["198.51.100.5:20200", "10.0.0.5:20200"]

service_locator:
# The address of the Service Locator (host:port).
address: "10.0.0.5:20000"

# The account username used by the Definitions Server to authenticate with the Service Locator.
username: definitions_service

# The password for the Definitions Server's Service Locator account.
password: admin_password

# Minimum version of the Spire client required to connect.
minimum_spire_version: "1"

# Name of the organization operating the platform.
organization: "Spire Trading Inc."

# NTP pool servers for time synchronization.
ntp_pool: ["0.pool.ntp.org:123",
"1.pool.ntp.org:123",
"2.pool.ntp.org:123",
"3.pool.ntp.org:123"]

# Paths to reference data files.
countries: countries.yml
currencies: currencies.yml
destinations: destinations.yml
venues: venues.yml
time_zones: date_time_zonespec.csv

# Exchange rate definitions.
exchange_rates:
- symbol: USD/CAD
rate: 1.33
- symbol: USD/AUD
rate: 1.49
- symbol: AUD/CAD
rate: 0.89
- symbol: HKD/CAD
rate: 0.17
...
</pre>

== Reference Data Files ==
The Definitions Server loads reference data from the following files:

=== countries.yml ===
Defines country codes according to ISO 3166 standard:
<pre>
---
countries:
- name: Canada
two_letter_code: CA
three_letter_code: CAN
code: 124
- name: United States
two_letter_code: US
three_letter_code: USA
code: 840
...
</pre>

A utility script (<code>update_countries.sh</code>) is provided to download and update country definitions from the official ISO 3166 data source. The script uses <code>country_code_parser.py</code> to parse the CSV data and generate the YAML file.

=== currencies.yml ===
Defines currency codes, symbols, and identifiers:
<pre>
---
currencies:
- code: CAD
sign: $
id: 124
- code: USD
sign: $
id: 840
...
</pre>

=== venues.yml ===
Defines trading venues (exchanges and markets) including their country, time zone, and currency:
<pre>
---
venues:
- venue: XTSE
country_code: CA
time_zone: Eastern_Time
currency: CAD
description: Toronto Stock Exchange
display_name: TSX
- venue: XASX
country_code: AU
time_zone: Australian_Eastern_Standard_Time
currency: AUD
description: Australian Stock Market
display_name: ASX
...
</pre>

=== destinations.yml ===
Defines order routing destinations and their applicable venues:
<pre>
---
destinations:
- id: TSX
description: Toronto Stock Exchange
venues:
- XTSE
- XTSX
- id: NEOE
description: Aequitas NEO Exchange
venues:
- NEOE
- XCNQ
- XTSE
- XTSX

# Preferred destinations specify default routing per venue
preferred_destinations:
- venue: XTSE
destination: TSX
- venue: NEOE
destination: NEOE

# Manual order entry destination
manual_order_entry:
id: MOE
description: Manual Order Entry
venues:
- NEOE
- XASX
- XCNQ
- XTSE
- XTSX
...
</pre>

=== date_time_zonespec.csv ===
Contains time zone database information in CSV format for accurate timestamp conversion across regions.

=== trading_schedules.yml ===
Defines market hours, holidays, and trading events:
<pre>
---
trading_schedules:
# Define closed days by weekday
- venues: [XASX]
time:
weekdays: [Sat, Sun]

# Define specific holiday dates
- venues: [XASX]
dates: [2020-01-01, 2020-04-10, 2020-12-25]

# Define trading session times
- venues: XASX
events:
- type: PRE_OPEN
time: 7:00
- type: OPEN
time: 10:00
- type: CLOSE
time: 16:00
...
</pre>

== Installation & Setup ==
A <code>setup.py</code> script is provided to generate the final <code>config.yml</code> from the <code>config.default.yml</code> template. The script also copies default reference data files if they do not already exist.

The script supports the following arguments:

<pre>
python setup.py
--local 0.0.0.0 # Local interface (default: auto-detected IP)
--world 198.51.100.5 # Global/public interface (optional)
--address 10.0.0.5:20000 # Service Locator address (default: local_interface:20000)
--password [REQUIRED] # Service password for authentication
</pre>

== Operations ==
The Definitions Server is controlled using three operational scripts: <code>start.sh</code>, <code>stop.sh</code>, and <code>check.sh</code>.

Log files are generated in the format:
<pre>
srv_YYYYMMDD_HH_MM_SS.log
</pre>

Upon startup, older log files are moved into the <code>logs/</code> directory.

=== check.sh ===
The <code>check.sh</code> script verifies whether the server is currently running by inspecting the PID recorded in <code>pid.lock</code> and testing whether the associated process exists.

If the server is not running, it prints:
<pre>
DefinitionsServer is not running.
</pre>

=== start.sh ===
The <code>start.sh</code> script:
* Exits immediately if the server is already running
* Creates a <code>logs/</code> directory if necessary
* Moves any existing <code>srv_*.log</code> files into <code>logs/</code>
* Starts the <code>DefinitionsServer</code> process in the background
* Writes the PID to <code>pid.lock</code>
* Reads network interfaces from <code>config.yml</code> and waits until the server is listening on at least one configured address

This ensures the server is fully initialized before the script exits.

=== stop.sh ===
The <code>stop.sh</code> script:
* Reads the PID from <code>pid.lock</code>
* Sends <code>SIGINT</code> to request a graceful shutdown
* Waits for termination using exponential backoff (up to 300 seconds)
* Sends <code>SIGKILL</code> if the server fails to stop cleanly
* Appends a forced-termination message to the most recent log file (if applicable)
* Removes the <code>pid.lock</code> file

This guarantees consistent shutdown behavior across normal and exceptional conditions.

Compliance Server

2025-12-01T21:59:57Z

Kamal: Created page with "= Compliance Server = The Compliance Server manages compliance rules and monitors rule violations for trading accounts. It maintains a repository of compliance rule definition..."

= Compliance Server =
The Compliance Server manages compliance rules and monitors rule violations for trading accounts. It maintains a repository of compliance rule definitions organized by directory entry, tracks rule states (active, passive, or deleted), and records violation events. By centralizing compliance rule administration and violation tracking, the server enforces consistent regulatory and risk management policies across the trading platform.

The Compliance Server integrates with the Service Locator for authentication and permission verification, the Administration Server to validate administrator privileges, and a MySQL database to persist compliance rules and violation records.

== Configuration ==
The Compliance Server is configured via a YAML file that defines network interfaces, database connection, and Service Locator integration. Below is the structure of the configuration file with example values:

<pre>
---
server:
# Primary network interface and port the Compliance Server binds to.
interface: "0.0.0.0:21900"

# List of addresses the server is reachable at (for registration with Service Locator).
# Typically includes both public-facing and local addresses.
addresses: ["198.51.100.5:21900", "10.0.0.5:21900"]

data_store:
# The address of the MySQL server (host:port).
address: "127.0.0.1:3306"

# The username for authenticating with MySQL.
username: spireadmin

# The password for the MySQL user.
password: 1234

# The name of the database schema where data is stored.
schema: spire

service_locator:
# The address of the Service Locator (host:port).
address: "10.0.0.5:20000"

# The account username used by the Compliance Server to authenticate with the Service Locator.
username: compliance_service

# The password for the Compliance Server's Service Locator account.
password: admin_password
...
</pre>

== Installation & Setup ==
A <code>setup.py</code> script is provided to generate the final <code>config.yml</code> from the <code>config.default.yml</code> template.

The script supports the following arguments:

<pre>
python setup.py
--local 0.0.0.0 # Local interface (default: auto-detected IP)
--world 198.51.100.5 # Global/public interface (optional)
--address 10.0.0.5:20000 # Service Locator address (default: local_interface:20000)
--password [REQUIRED] # Service password for authentication
--mysql_address 127.0.0.1:3306 # MySQL server address
--mysql_username spireadmin # MySQL username
--mysql_password secretpw # MySQL password (default: --password if omitted)
--mysql_schema spire # MySQL schema
</pre>

== Functionality ==
The Compliance Server provides the following capabilities:

=== Rule Management ===
Compliance rules are organized by directory entry (accounts or groups) and consist of:
* A unique rule identifier
* The associated directory entry
* A state (active, passive, or deleted)
* A compliance rule schema defining the rule logic

Administrators can create, update, and delete compliance rules. Rule changes are immediately propagated to subscribed clients for real-time enforcement.

=== Rule Monitoring ===
Clients with appropriate permissions can:
* Load existing compliance rules for a specific directory entry
* Subscribe to receive real-time updates when rules are added, modified, or deleted

Subscriptions enable clients to maintain synchronized views of active compliance rules without polling.

=== Violation Reporting ===
Administrators can report compliance rule violations, which are:
* Timestamped by the server
* Persistently stored in the database
* Available for audit and analysis

== Operations ==
The Compliance Server is controlled using three operational scripts: <code>start.sh</code>, <code>stop.sh</code>, and <code>check.sh</code>.

Log files are generated in the format:
<pre>
srv_YYYYMMDD_HH_MM_SS.log
</pre>

Upon startup, older log files are moved into the <code>logs/</code> directory.

=== check.sh ===
The <code>check.sh</code> script verifies whether the server is currently running by inspecting the PID recorded in <code>pid.lock</code> and testing whether the associated process exists.

If the server is not running, it prints:
<pre>
ComplianceServer is not running.
</pre>

=== start.sh ===
The <code>start.sh</code> script:
* Exits immediately if the server is already running
* Creates a <code>logs/</code> directory if necessary
* Moves any existing <code>srv_*.log</code> files into <code>logs/</code>
* Starts the <code>ComplianceServer</code> process in the background
* Writes the PID to <code>pid.lock</code>
* Reads network interfaces from <code>config.yml</code> and waits until the server is listening on at least one configured address

This ensures the server is fully initialized before the script exits.

=== stop.sh ===
The <code>stop.sh</code> script:
* Reads the PID from <code>pid.lock</code>
* Sends <code>SIGINT</code> to request a graceful shutdown
* Waits for termination using exponential backoff (up to 300 seconds)
* Sends <code>SIGKILL</code> if the server fails to stop cleanly
* Appends a forced-termination message to the most recent log file (if applicable)
* Removes the <code>pid.lock</code> file

This guarantees consistent shutdown behavior across normal and exceptional conditions.

Charting Server

2025-12-01T21:52:36Z

Kamal: Created page with "= Charting Server = The Charting Server processes time series queries on market data and publishes ongoing updates to clients. It performs calculations such as candlestick agg..."

= Charting Server =
The Charting Server processes time series queries on market data and publishes ongoing updates to clients. It performs calculations such as candlestick aggregation, technical indicators, and other analytical transformations over historical and real-time market data. By centralizing these computations, the server enables multiple clients to subscribe to identical chart data without redundant calculation overhead.

The Charting Server integrates with the Service Locator for authentication and service registration. Multiple Charting Servers can be deployed concurrently to distribute query processing load across instances with the Service Locator performing the load balancing among them.

== Configuration ==
The Charting Server is configured via a YAML file that defines its network interface and Service Locator integration. Below is the structure of the configuration file with example values:

<pre>
---
server:
# Primary network interface and port the Charting Server binds to.
interface: "0.0.0.0:21400"

# List of addresses the server is reachable at (for registration with Service Locator).
# Typically includes both public-facing and local addresses.
addresses: ["198.51.100.5:21400", "10.0.0.5:21400"]

service_locator:
# The address of the Service Locator (host:port).
address: "10.0.0.5:20000"

# The account username used by the Charting Server to authenticate with the Service Locator.
username: charting_service

# The password for the Charting Server's Service Locator account.
password: admin_password
...
</pre>

== Installation & Setup ==
A <code>setup.py</code> script is provided to generate the final <code>config.yml</code> from the <code>config.default.yml</code> template.

The script supports the following arguments:

<pre>
python setup.py
--local 0.0.0.0 # Local interface (default: auto-detected IP)
--world 198.51.100.5 # Global/public interface (optional)
--address 10.0.0.5:20000 # Service Locator address (default: local_interface:20000)
--password [REQUIRED] # Service password for authentication
</pre>

== Operations ==
The Charting Server is controlled using three operational scripts: <code>start.sh</code>, <code>stop.sh</code>, and <code>check.sh</code>.

Log files are generated in the format:
<pre>
srv_YYYYMMDD_HH_MM_SS.log
</pre>

Upon startup, older log files are moved into the <code>logs/</code> directory.

=== check.sh ===
The <code>check.sh</code> script verifies whether the server is currently running by inspecting the PID recorded in <code>pid.lock</code> and testing whether the associated process exists.

If the server is not running, it prints:
<pre>
ChartingServer is not running.
</pre>

=== start.sh ===
The <code>start.sh</code> script:
* Exits immediately if the server is already running
* Creates a <code>logs/</code> directory if necessary
* Moves any existing <code>srv_*.log</code> files into <code>logs/</code>
* Starts the <code>ChartingServer</code> process in the background
* Writes the PID to <code>pid.lock</code>
* Reads network interfaces from <code>config.yml</code> and waits until the server is listening on at least one configured address

This ensures the server is fully initialized before the script exits.

=== stop.sh ===
The <code>stop.sh</code> script:
* Reads the PID from <code>pid.lock</code>
* Sends <code>SIGINT</code> to request a graceful shutdown
* Waits for termination using exponential backoff (up to 300 seconds)
* Sends <code>SIGKILL</code> if the server fails to stop cleanly
* Appends a forced-termination message to the most recent log file (if applicable)
* Removes the <code>pid.lock</code> file

This guarantees consistent shutdown behavior across normal and exceptional conditions.

Administration Server

2025-12-01T21:45:00Z

Kamal: Created page with "= Administration Server = The Administration Server manages account-level authorization and operational metadata for Spire. It maintains each account's role (trader, manager,..."

= Administration Server =

The Administration Server manages account-level authorization and operational metadata for Spire. It maintains each account's role (trader, manager, administrator, or service), market-data entitlements, risk parameters, and trading permissions. By centralizing these attributes, it enforces access control, entitlement visibility, and trading eligibility across the system.
The server integrates with the Service Locator to enumerate accounts and with a MySQL database to persist administrative data.

== Configuration ==
The Administration Server is configured via a YAML file that defines network interfaces, database connection, Service Locator integration, and market-data entitlements. Below is the structure of the configuration file with example values:

<pre>
---
server:
# Primary network interface and port the Administration Server binds to.
interface: "0.0.0.0:20100"

# List of addresses the server is reachable at (for registration with Service Locator).
# Typically includes both public-facing and local addresses.
addresses: ["198.51.100.5:20100", "10.0.0.5:20100"]

data_store:
# The address of the MySQL server (host:port).
address: "127.0.0.1:3306"

# The username for authenticating with MySQL.
username: spireadmin

# The password for the MySQL user.
password: 1234

# The name of the database schema where data is stored.
schema: spire

service_locator:
# The address of the Service Locator (host:port).
address: "10.0.0.5:20000"

# The account username used by the Administration Server to authenticate with the Service Locator.
username: administration_service

# The password for the Administration Server's Service Locator account.
password: admin_password

entitlements:
# Market-data entitlement definitions.
# Each entitlement specifies pricing, applicable exchanges, and message types.
- name: Demo Entitlements
price: 0.00
currency: USD
group: demo_entitlements
applicability:
# source: the venue disseminating the market data
# messages: market data message types
# BBO - best bid offer quotes
# BOOK - book quotes
# TAS - time and sales
# IMB - order imbalances
- source: CHIC
messages: [BBO, BOOK, TAS, IMB]
- source: XTSE
messages: [BBO, BOOK, TAS, IMB]
- source: XTSX
messages: [BBO, BOOK, TAS, IMB]
...
</pre>

== Installation & Setup ==

A <code>setup.py</code> script is provided to generate the final <code>config.yml</code>.
The script supports the following arguments:

<pre> python setup.py
--local 0.0.0.0 # Local interface (default: auto-detected IP)
--world 198.51.100.5 # Global/public interface (optional)
--address 10.0.0.5:20000 # Service Locator address (default: local_interface:20000)
--password [REQUIRED] # Administration password
--mysql_address 127.0.0.1:3306 # MySQL server address
--mysql_username spireadmin # MySQL username
--mysql_password secretpw # MySQL password (default: --password if omitted)
--mysql_schema spire # MySQL schema
</pre>

== Risk State Initialization Utility ==

A reset script (<code>reset_risk_states.py</code>) is provided to initialize or reset the risk state for all accounts and is typically run on a daily basis.

== Operations ==

The Administration Server is controlled using three operational scripts: <code>start.sh</code>, <code>stop.sh</code>, and <code>check.sh</code>.
Log files are generated in the format:

<pre> srv_YYYYMMDD_HH_MM_SS.log </pre>

Upon startup, older log files are moved into the <code>logs/</code> directory.

=== check.sh ===

The <code>check.sh</code> script verifies whether the server is currently running by inspecting the PID recorded in <code>pid.lock</code> and testing whether the associated process exists.
If the server is not running, it prints:

<pre> AdministrationServer is not running. </pre>

=== start.sh ===

The <code>start.sh</code> script:

Exits immediately if the server is already running

Creates a <code>logs/</code> directory if necessary

Moves any existing <code>srv_*.log</code> files into <code>logs/</code>

Starts the <code>AdministrationServer</code> process in the background

Writes the PID to <code>pid.lock</code>

Reads network interfaces from <code>config.yml</code> and waits until the server is listening on at least one configured address

This ensures the server is fully initialized before the script exits.

=== stop.sh ===

The <code>stop.sh</code> script:

Reads the PID from <code>pid.lock</code>

Sends <code>SIGINT</code> to request a graceful shutdown

Waits for termination using exponential backoff (up to 300 seconds)

Sends <code>SIGKILL</code> if the server fails to stop cleanly

Appends a forced-termination message to the most recent log file (if applicable)

Removes the <code>pid.lock</code> file

This guarantees consistent shutdown behavior across normal and exceptional conditions.

NEOE Market Data Feed Client

2025-02-25T16:32:10Z

Kamal: Created page with "The NEOE Market Data Feed Client disseminates real-time market data for the Aequitas NEO Exchange. It connects to the Service Locator for authentication and provides order boo..."

The NEOE Market Data Feed Client disseminates real-time market data for the Aequitas NEO Exchange. It connects to the Service Locator for authentication and provides order book data, time and sales, and the consolidated best bid offer for the NEO Exchange. The feed is managed using start.sh, stop.sh, and check.sh scripts for service control.

==Configuration==

The feed is split into multiple channels for disseminating the CBBO, book, and trade data. Each channel can be found in its own subdirectory whose name starts with the neoe prefix, such as neoe_cbbo, neoe_cls and neoe_book for the CBBO, time and sales, and book feed respectively.

The feed is configured via a YAML file (<code>config.yml</code>) with the following structure and example values:

<pre>
---
service_locator:
# Service Locator connection details
address: "127.0.0.1:20000"
username: "market_data_feed"
password: "securepassword123"

# Whether to log raw market data packets
enable_logging: false

# Path to security definitions (relative to config)
symbol_list: "../symbols.yml"

# Frequency of market data updates
sampling: 100ms

# Retransmission settings
enable_retransmission: false
max_retransmissions: 10000
retransmission_block_size: 20000

# Multicast network configuration
host: "233.102.209.247:61023"
interface: "192.168.0.100:61023"
retransmission_request_address: "142.201.149.44:61030"
retransmission_response_address: "142.201.149.44:61031"

# Market Participant ID mappings
mpid_mappings:
- source: 1
name: ANON
- source: 2
name: RBCC
- source: 3
name: TRIS
- source: 4
name: VERS
...
</pre>

In addition there is a security master list of traded securities typically founds in the <code>symbols.yml</code> file with the following format:

<pre>
---
- symbol: AAPL # The ticker symbol assign.
name: APPLE CDR (CAD HEDGED) # The name of the security.
board_lot: 100 # The security's board lot.
- symbol: ABXX
name: ABAXX TECHNOLOGIES INC.
board_lot: 100
- symbol: ABXX.WT
name: ABAXX TECHNOLOGIES INC. WARRANTS
board_lot: 100
...
</pre>

Changes can be made in this security master list and reloaded by restarting the NEOE CBBO feed.

==Installation & Setup==

===<code>setup.py</code> Script===

The <code>setup.py</code> Python script configures all of the channels at once and can be called as follows:

<pre>
python3 setup.py
--local 192.168.0.100 # Local interface (default: auto-detected IP)
--address 127.0.0.1:20000 # Service Locator address
--username market_data_feed # Service Locator username (default: market_data_feed)
--password [REQUIRED] # Service Locator password
</pre>

==Operations==

Log files are generated as srv_[YYYY][MM][DD]_[HH]_[MM]_[SS].log in the runtime directory, upon termination, non-empty log files are moved into the log/ subfolder, while empty log files are deleted.

Each channel is managed through three core scripts:

start.sh Starts the channel, does nothing if the channel is already running.

stop.sh Stops the channel and waits for the channel to terminate.

check.sh Verifies that the channel is running.

Uid Server

2025-02-05T00:20:32Z

Kamal: Created page with "The Uid Server is a centralized service responsible for generating batches of unique integers that can be used to identify items, such as database entries. The Uid Server regi..."

The Uid Server is a centralized service responsible for generating batches of unique integers that can be used to identify items, such as database entries. The Uid Server registers itself with the Service Locator as a <code>uid_server</code> and ensures uniqueness by tracking allocated identifiers in a MySQL database.

==Configuration==

The Uid Server is configured via a YAML file that defines its network interface, data store settings, and Service Locator integration. Below is the structure of the configuration file with example values:

<pre>
---
server:
# The network interface and port the Uid Server binds to.
interface: "0.0.0.0:20900"

# List of addresses the server advertises for client connections.
addresses: ["192.168.1.100:20900", "0.0.0.0:20900"]

data_store:
# MySQL server address (host:port).
address: "127.0.0.1:3306"

# MySQL credentials.
username: uid_server
password: securepassword123
schema: uid_db

service_locator:
# Service Locator address for registration.
address: "127.0.0.1:20000"

# Uid Server credentials for Service Locator.
username: uid_service
password: adminpassword123
...
</pre>

==Installation & Setup==

A Python script automates configuration file generation. Usage:

<pre>
python setup.py
--local 0.0.0.0 # Local interface (default: auto-detected IP)
--world 192.168.1.100 # Global interface (default: same as local)
--address 127.0.0.1:20000 # Service Locator address
--password [REQUIRED] # Admin password for Service Locator
--mysql_address 127.0.0.1:3306 # MySQL server address
--mysql_username admin # MySQL username (default: spireadmin)
--mysql_password [OPTIONAL] # MySQL password (default: same as admin password)
--mysql_schema uid_db # Database schema (default: spire)
</pre>

Features:

* Generates config.yml from config.default.yml template.
* Requires only the --password argument by default.
* Auto-detects local IP if --local is omitted.

==Operations==

Log files are generated as srv_[YYYY][MM][DD]_[HH]_[MM]_[SS].log in the runtime directory, upon termination, non-empty log files are moved into the log/ subfolder, while empty log files are deleted.

The Uid Server is managed through three core scripts:

start.sh
Starts the Uid server, does nothing if the server is already running.

stop.sh
Stops an existing server instance and waits for the server to terminate.

check.sh
Verifies that the server is running.

Service Locator

2025-02-04T19:55:47Z

Kamal:

The Service Locator is a centralized server responsible for authenticating account logins, creating and managing user accounts, organizing accounts into directories, controlling permissions, and registering available services. It acts as a critical component for coordinating access control and service discovery. The Service Locator integrates with a MySQL database to persistently store account details, directory structures, permissions, and service registration data.

==Configuration==

The Service Locator is configured via a YAML file that defines its network interface and data store settings. Below is the structure of the configuration file with example values:

<pre>
---
# The network interface and port the Service Locator binds to.
interface: "0.0.0.0:20000"

data_store:
# The address of the MySQL server (host:port).
address: "127.0.0.1:3306"

# The username for authenticating with MySQL.
username: service_locator

# The password for the MySQL user.
password: 1234

# The name of the database schema where data is stored.
schema: spire
...
</pre>

==Installation & Setup==

A <code>setup.py</code> script is provided for convenience to configure the Service Locator and generate the <code>config.yml</code> file. The script supports the following arguments:

<pre>
python setup.py
--local 0.0.0.0 # Local interface (default: auto-detected IP)
--mysql_address 127.0.0.1:3306 # MySQL server address
--mysql_username admin # MySQL username (default: spireadmin)
--mysql_password [REQUIRED] # MySQL password (no default)
--mysql_schema service_db # Database schema (default: spire)
</pre>

==Operations==

Log files are generated as srv_[YYYY][MM][DD]_[HH]_[MM]_[SS].log in the runtime directory, upon termination, non-empty log files are moved into the log/ subfolder, while empty log files are deleted.

The Service Locator is managed through three core scripts:

start.sh
Starts the Service Locator server, does nothing if the server is already running.

stop.sh
Stops an existing server instance and waits for the server to terminate.

check.sh
Verifies that the server is running.

Service Locator

2025-02-04T19:54:58Z

Kamal: Created page with "The Service Locator is a centralized server responsible for authenticating account logins, creating and managing user accounts, organizing accounts into directories, controlli..."

The Service Locator is a centralized server responsible for authenticating account logins, creating and managing user accounts, organizing accounts into directories, controlling permissions, and registering available services. It acts as a critical component for coordinating access control and service discovery. The Service Locator integrates with a MySQL database to persistently store account details, directory structures, permissions, and service registration data.

==Configuration==

The Service Locator is configured via a YAML file that defines its network interface and data store settings. Below is the structure of the configuration file with example values:

<pre>
---
# The network interface and port the Service Locator binds to.
interface: "0.0.0.0:20000"

data_store:
# The address of the MySQL server (host:port).
address: "127.0.0.1:3306"

# The username for authenticating with MySQL.
username: service_locator

# The password for the MySQL user.
password: securepassword123

# The name of the database schema where data is stored.
schema: service_locator_db
...
</pre>

==Installation & Setup==

A <code>setup.py</code> script is provided for convenience to configure the Service Locator and generate the <code>config.yml</code> file. The script supports the following arguments:

<pre>
python setup.py
--local 0.0.0.0 # Local interface (default: auto-detected IP)
--mysql_address 127.0.0.1:3306 # MySQL server address
--mysql_username admin # MySQL username (default: spireadmin)
--mysql_password [REQUIRED] # MySQL password (no default)
--mysql_schema service_db # Database schema (default: spire)
</pre>

==Operations==

Log files are generated as srv_[YYYY][MM][DD]_[HH]_[MM]_[SS].log in the runtime directory, upon termination, non-empty log files are moved into the log/ subfolder, while empty log files are deleted.

The Service Locator is managed through three core scripts:

start.sh
Starts the Service Locator server, does nothing if the server is already running.

stop.sh
Stops an existing server instance and waits for the server to terminate.

check.sh
Verifies that the server is running.

C++ Style Guide

2018-02-23T01:55:10Z

Kamal: /* Development Environment */

This article specifies the most general guidelines used for all Spire projects written using C++ as the primary programming language. Each individual project may extend or override the guidelines specified herein. The overall aim of this guideline is to provide consistency across the numerous projects developed by Spire in order to promote and benefit from modern C++ practices as well as tailor them to our specific requirements.

In general, the guidelines are very specific and opinionated. This is done intentionally to ensure that every detail has been given proper consideration and that every line of code is written with care and diligence. It is taken as a prerequisite that writing robust, clean and maintainable code requires paying close attention to even the smallest of details. As such, when there are two or more ways to write or express any given statement or expression, be it using spaces or tabs, number of characters per line, purple or violet, the guideline will often require that one particular approach be used to the exclusion of others.

Finally, this document is constantly evolving and as such older projects may not be up-to-date with the guidelines found here. In those circumstances one must use their best judgment about how to integrate these guidelines into older codebases. As a general principle, consistency should favor the local over the global, that is it is more important to be consistent with a function definition than within a file, and more important to be consistent within a file than within a directory, project, etc...

=Core Guidelines=
This document should be seen as an extension to the Cpp Core Guidelines authored and maintained by Herb Sutter and Bjarne Stroustrup. After reviewing these guidelines one should then become familiar with the Cpp Core Guidelines as documented here:

https://github.com/isocpp/CppCoreGuidelines

Those guidelines specify the best practices for writing modern C++ code. In situations where there is a conflict between the Cpp Core Guidelines and the guidelines set forth in this document, this document takes precedence. For all known situations where a conflict exists, this document should explicitly indicate that conflict to avoid confusion.

=Development Environment=

The two primary development environments supported by Spire projects are Ubuntu Server 16.04 LTS and Windows 10. On Unix like systems the compiler of choice is g++ 5.4 and on Windows 10 it's Microsoft Visual Studio 2017 15.5.7. In order to support these two platforms, CMake is used to produce the appropriate project/make files for each platform.

For source control, git is used and projects are to be hosted on [https://github.com/spiretrading Spire Trading's Github page].

Each developer is welcome to use an editor of their choice. [https://www.visualstudio.com/downloads/ Visual Studio 2017] is typically used for Windows and [https://code.visualstudio.com/ Visual Studio Code] is typically used on Linux and Mac.

=File Names=

Use [https://en.wikipedia.org/wiki/Snake_case snake case] using all lower case letters for files and directories. The main exception is for CMake files which must use the name ''CMakeLists.txt''

Header files should use the extension ''hpp''

Source files should use the extension ''cpp''

All files end with a single new line character.

=Directory Structure=

A project is broken down into one library component and one or more application components. These components are referred to as ''artifacts''. The library is written in a [https://en.wikipedia.org/wiki/Header-only header-only fashion], that is the implementation is provided directly in the header file rather than a separate .cpp source file. This allows for such libraries to easily be incorporated into other projects without the need for complex build configurations. Applications often use a mix of header only files and source files.

The applications go into their own ''applications'' directory and the libraries into the ''library'' directory. Within each artifact is a ''build'' directory containing the build scripts and CMake files needed to build that artifact. Scripts for POSIX systems are found in the ''posix'' sub-directory and Windows build files are within the ''windows'' sub-directory. The build scripts specified are ''run_cmake'' to produce the platform specific build files (make files for POSIX and VS solutions for Windows), ''version'' to produce a header file that contains the version of the artifact produced and the ''build'' script which produces the artifact. In order to build an artifact, one first produces the build files for their platform by running the ''run_cmake'' script, and then they can run the ''build'' script afterwards to produce the artifact.

Finally there is a top-level ''build'' directory for the project as a whole which produces all artifacts by recursively calling each artifact's individual build scripts. Additionally the top-level build directory also includes a ''setup'' script which downloads all third party dependencies. Some projects may also opt to include an ''install'' script which sets up a suitable development environment, and other scripts for continuous builds, deployment and testing.

Assuming a project named tic_tac_toe consisting of applications tic_tac_toe_console and tic_tac_toe_ui sharing code common to both applications, the following directory structure should be used:

<pre>
tic_tac_toe # Root level directory.
\applications # Contains all applications.
\tic_tac_toe_console # Contains the source for a console application.
\build # Files/scripts used to build the console tic-tac-toe game.
\config # Contains all CMake build files.
CMakeLists.txt # The CMake config file for the overall application.
\tic_tac_toe_console # Contains CMake build files for the main executable.
\CMakeLists.txt # The CMake config file for the executable.
\posix # Contains scripts to build on POSIX systems.
\build.sh # Script used to build the application.
\run_cmake.sh # Script used to produce the make/build files.
\set_env.sh # Script defining the environment variables.
\version.sh # Script that produces the VERSION stamped header file.
\windows # Contains scripts to build on Windows.
\build.bat # Script used to build the application.
\run_cmake.bat # Script used to produce the VS solution.
\set_env.bat # Script defining environment variables.
\version.bat # Script that produces the VERSION stamped header file.
\source # Contains the C++ source files.
\tic_tac_toe_console # Contains the file defining the main function.
\main.cpp # Defines the main function.
\source_dir_a # Contains additional source files specific the console.
\source_a.cpp # C++ source code files.
\source_b.cpp
\source_c.cpp
\source_dir_b # Contains additional source files specific the console.
\source_d.cpp
\source_e.cpp
\source_f.cpp
\tic_tac_toe_ui # Contains a GUI version of tic-tac-toe.
\... # The structure is similar to the console application.
\build # Contains build scripts to build the entire project.
\posix # Contains scripts to build on POSIX systems.
\build.sh # Builds all libraries and applications.
\local_build.sh # A helper script containing common build functions.
\run_cmake.sh # Produces all project make/build files.
\setup.sh # Installs all third party dependencies.
\windows # Contains scripts to build on Windows.
\build.bat # Builds all libraries and applications.
\run_cmake.bat # Produces all VS solutions.
\setup.bat # Installs all third party dependencies.
\library # Contains the common library code.
\build # Files/script used to build the library.
\config # Contains all CMake build files.
\CMakeLists.txt # The CMake config file for the library.
\dir_a # Contains CMake build files for library component A.
\CMakeLists.txt # The CMake config file to build component A and unit tests.
\dir_b # Contains CMake build files for library component B.
\CMakeLists.txt # The CMake config file to build component B and unit tests.
\posix # Contains scripts to build on POSIX systems.
\build.sh # Script used to build the library.
\run_cmake.sh # Script used to produce make/build files.
\set_env.sh # Script defining the environment variables.
\windows # Contains scripts to build on Windows.
\build.bat # Script to build the library.
\run_cmake.bat # Script to produce the VS solution.
\set_env.bat # Script defining the environment variables.
\include # Contains the library header files (.hpp)
\tic_tac_toe
\tic_tac_toe # Contains the top-most generic library header files.
\tic_tac_toe.hpp # Header file containing generic forward declarations.
\dir_a # Contains header files for library component A.
\a.hpp # Header file containing forward declarations for component A.
\a_header_1.hpp # Header file for library component A.
\a_header_2.hpp
\a_header_3.hpp
\dir_b
\b.hpp # Header file containing forward declarations for component B.
\b_header_1.hpp # Contains header files for library component B.
\b_header_2.hpp
\b_header_3.hpp
\source # Contains the library source files (.cpp)
\tic_tac_toe # Dummy directory for the overall library.
\dummy.cpp # Dummy source code file.
\dir_a # Dummy directory for library component A.
\dummy.cpp # Dummy source file.
\dir_a_tester # Contains unit tests for library component A.
\a_header_1_tester.cpp # Contains unit tests for definitions in a_header_1.hpp
\a_header_2_tester.cpp
\a_header_3_tester.cpp
\main.cpp # The entry point for component A unit tests.
\dir_b # Dummy directory for library component B.
\dummy.cpp # Dummy source file.
\dir_b_tester # Contains unit tests for library component B.
\b_header_1_tester.cpp # Contains unit tests for definitions in b_header_a.hpp
\b_header_2_tester.cpp
\b_header_3_tester.cpp
\main.cpp # The entry point for component B unit tests.
</pre>

An example of the above directory structure containing a basic skeletal implementation of all files can be found here: https://github.com/spiretrading/tic_tac_toe

=Header File Structure=

==Include Guard==
Header files are structured first with an [https://en.wikipedia.org/wiki/Include_guard include guard] defined using capital letter snake case whose name consists of the project name appended to the directory name and finally the file name itself. As a note, all files end with a single new line character.
<syntaxhighlight lang=c++ line=line>
#ifndef CPP_CHAT_SERVER_HPP
#define CPP_CHAT_SERVER_HPP
...
#endif
</syntaxhighlight>

==Include Directives==
Next come the list of #include directives. Include files are ordered based on their category where the first category is standard C++ header files, the second category is external dependency header files, and finally the third category is project header files. Both standard and external dependency header files use angle brackets (#include <...>) and local project header files use quotes (#include "..."). Within each category files are listed in alphabetical order.
<syntaxhighlight lang=c++ line=line>
#include <tuple>
#include <vector>
#include <boost/noncopyable.hpp>
#include "cpp_chat/cpp_chat.hpp"
#include "cpp_chat/definitions.hpp"
</syntaxhighlight>

==Namespaces==
After include directives the project's namespace is defined. All declarations and definitions must be contained within a namespace so as to avoid polluting the global namespace. Namespace definitions should be preceded by one single new line except when immediately following a namespace definition. The top level namespace is named after the project, and sub-namespaces may be used (although they should be used very sparingly).

<syntaxhighlight lang=c++ line=line>
namespace cpp_chat {
namespace sub_chat_a {
...
}

namespace sub_chat_b {
namespace sub_sub_chat_a {
...
}
}
}

namespace cpp_chat_extra {
...
}
</syntaxhighlight>

One situation where nested namespaces are welcome and encouraged are for implementation details that do not form part of the namespace's public interface. These definitions are put in a nested namespace called ''details'' and go into a file of their own whose name contains the suffix ''details''. For example if the ''cpp_chat_server.hpp'' contains implementation details, then they should be included in the file ''cpp_chat_server_details.hpp'' and the definitions should go into the namespace ''cpp_chat::details''.

=Layout=

==Indentation==
Code is indented using 2 spaces per level, tabs are not permitted.

<syntaxhighlight lang=c++ line=line>
int factorial(int n) {
if(n == 0) {
return 1;
}
return n * factorial(n - 1);
}
</syntaxhighlight>

==Line Structure==
Lines are limited to 80 characters. Exceptions to this rule are long include files and long string literals where breaking up the literal into multiple lines is not possible. In order to break up long statements into multiple lines the following rules are used:

* Break the line at a comma, operator or opening bracket.
<syntaxhighlight lang=c++ line=line>

// Correct, line break at comma.
f(a, ..., b,
c, ..., d);

// Incorrect, line break after expression.
f(a, ..., b
, c, ..., d);

// Correct, line break after operator.
a + ... + b +
c + ... + d;

// Incorrect, line break after expression.
a + ... + b
+ c + ... + d;

// Correct, line break after opening bracket.
f(
a, ..., b, c, ..., d);

// Incorrect, line break after function name.
f
(a, ..., b, c, ..., d);
</syntaxhighlight>

* For statements that begin new blocks of code (such as if/while/class...), in order to avoid confusing the line continuation with the code block, the line continuation is indented two extra levels. For example consider the following snippet of code:

<syntaxhighlight lang=c++ line=line>
// Incorrect.
if(condition_a && condition_b && ... &&
condition_c) {
std::cout << "meow" << std::endl;
}
</syntaxhighlight>

The line continuation on line 2 clashes with the code block on line 3. To avoid this clash the above code is indented as follows:

<syntaxhighlight lang=c++ line=line>
// Correct.
if(condition_a && condition_b && ... &&
condition_c) {
std::cout << "meow" << std::endl;
}
</syntaxhighlight>

The extra level of indentation in the line continuation makes it clear where the condition ends and the code block begins.

==Braces==

Braces are placed using a variant of the [http://wiki.c2.com/?OneTrueBraceStyle OTBS style]. The opening brace is placed on the same line as the declaring statement with one single space preceding it, and the closing brace is placed on a line of its own at the same level of indentation as the declaring statement. For if statements, the else/else if is placed on the same line as the closing brace. For do/while loops, the while is placed on the same line as the closing brace.

Examples:
<syntaxhighlight lang=c++ line=line>
// Correct.
if(<cond>) {
<body>
} else {
<default>
}

// Correct.
do {
<body>
} while(<cond>);

// Incorrect.
if(<cond>)
{
<body>
}
else
{
<default>
}

// Incorrect.
do
{
<body>
}
while(<cond>);
</syntaxhighlight>

==Spacing==

The following are correct use cases for white spaces:

* Used for indentation.
* Used to surround binary operations.
<syntaxhighlight lang=c++ line=line>
// Correct.
auto x = a + b;
auto y = 5;

//! Incorrect
auto x = a+b;
auto y=5;
auto z= 5;
</syntaxhighlight>

* One space is placed after a comma.
<syntaxhighlight lang=c++ line=line>
// Correct.
f(1, 2);
int g(int a, int b);

//! Incorrect
f(1,2);
int g(int a,int b);
</syntaxhighlight>

=Syntax=

==Function Definitions==

Functions declared and defined in header files are formatted as follows:
<syntaxhighlight lang=c++ line=line>
inline int f() {
...
return 123;
}

template<typename T>
bool g() {
...
return false;
}
</syntaxhighlight>

That is they are declared as inline unless they are function templates in which case the inline specified is omitted.

==Variable Declarations==

Variables are declared so that only one variable is declared per line, the type of the variable should [https://herbsutter.com/2013/08/12/gotw-94-solution-aaa-style-almost-always-auto/ almost always use auto], and variables should almost always be initialized.

Examples of simple declarations:
<syntaxhighlight lang=c++ line=line>
auto x = 5;
auto y = a + b;
</syntaxhighlight>

For complex initialization of values, use an [https://en.wikipedia.org/wiki/Immediately-invoked_function_expression immediately invoked lambda expression] as follows:

<syntaxhighlight lang=c++ line=line>
auto value = [&] {
if(condition) {
return 123;
}
return 321;
}();
</syntaxhighlight>

=Additional References=

The bulk of this style guide focuses on syntax rather than good programming practices. The following list are reference materials (both online and published) for best practices on writing portable, efficient, and clean C++ code:

* S. Meyers, ''Effective Modern C++''. 2014
* B. Stroustrup ''The C++ Programming Language (4th Edition)''. 2013
* [https://github.com/isocpp/CppCoreGuidelines Cpp Core Guidelines]
* [https://isocpp.org/ Standard C++ Blog]
* [https://cppcon.org/ CppCon - The C++ Conference]

Query

2018-01-21T22:52:56Z

Kamal:

Beam provides a series of classes to query historical and real time streaming data along a wide range of search criteria. It is one of the core APIs provided by Beam and used extensively in Spire to search through and disseminate market data and order executions.

==Structure==
Much of the data stored by Spire is in the form of a time series. The data is grouped together based on an index, for example time and sales data is indexed by its ticker symbol, order's are indexed by the account the order belongs to etc... For every item there is both a timestamp and a sequence number associated with it (to identify which data was published first in the event that two items have the same timestamp).

The most common type of query used in Spire to search through a time series is defined by the Beam::Queries::BasicQuery class. It consists of five components that are used to fully specify the type of data that is to be retrieved. Those five components are described in detail as follows:

===Index===
Every data item to query over must be associated with a single index. Specifying the value of the index to query over is defined by the class Beam::Queries::IndexedQuery.

===Range===
This component is defined by the class Beam::Queries::RangedIndex and it specifies the time range to search through as well as whether the query is strictly for historical data, real time data, or both. The class Beam::Queries::Range defines the semantics of a range and consists of a start value and an end value, both of which can be either a timestamp or a sequence number. Sequence numbers are defined by the class Beam::Queries::Sequence and it contains two special values which control the behavior of a query: LAST and PRESENT.

If the end of a range is specified to be PRESENT then the query will return all matching values that are currently stored in the database. If the end of a range is specified to be LAST or +infinity then in addition to returning all currently matching values, the query will also return matching values published in real time as they arrive.

Any other value used to mark the end of a range results in a strictly historical data query, even if the value denotes some point in the future.

===Snapshot limit===
This specifies the maximusm number of historical data items to retrieve. This is typically used to prevent a query from returning too much data. The class Beam::Queries::SnapshotLimit contains two values, the size of the limit and the type of the limit. The size controls the maximum number of historical data items are returned by a query (with the special value of UNLIMITED if every value should be returned), as well as the type of snapshot. The type dictates whether items should be returned from the beginning of the series or the end of the series (the HEAD and TAIL respectively).

For example to query for the very last item currently stored in a time series, one can use a range of [FIRST, PRESENT] and a snapshot limit whose type is TAIL and whose size is 1. Alternatively to query for the first 10 values in a time series one would use a range of [FIRST, PRESENT] and a snapshot limit with type HEAD and size 10.

The class Beam::Queries::SnapshotLimitedQuery is the component used to indicate what the snapshot limit of a query should be. By default, this class specifies that no values are to be returned (its size is 0). This is because one is expected to be explicit about how much data is desired. The special value Beam::Queries::SnapshotLimit::Unlimited() can be used to explicitly indicate that there is no limit on how much data should be returned, however, it should be noted that the database engine handling the request may impose limits of its own regarding the maximum size of a snapshot.

===Interruption policy===
When querying for real time data it is possible for the query to get interrupted, usually due to a disconnection. This component, defined by Beam::Queries::InterruptableQuery, specifies what action should be taken in the event of such an interruption. The possible recovery options are defined by the enumerator Beam::Queries::InterruptionPolicy as follows:

RECOVER_DATA, the query should attempt to re-establish a connection to the database and recover any data that was published during the time that the query was interrupted.
IGNORE_CONTINUE, the query should attempt to re-establish a connection to the database but ignore any data that was published during the time that the query was interrupted. This is usually desirable when only live, real time data is desired as this option will omit any stale data published during the interruption.
BREAK_QUERY, the query should abort, indicating an exception. This is the default interruption policy.

===Filter===

==Example==
An example of a query for historical data might be to retrieve the first 1000 time and sales for MSFT.NSDQ from August 11th, 2016 to August 12th, 2016.

===C++===
<syntaxhighlight lang=c++ line=line>
#include <iostream>
#include "Nexus/MarketDataService/SecurityMarketDataQuery.hpp"
#include "Nexus/ServiceClients/ApplicationServiceClients.hpp"

int main() {
Beam::Network::SocketThreadPool socketThreadPool;
Beam::Threading::TimerThreadPool timerThreadPool;

// Connect to the service.
Nexus::ApplicationServiceClients serviceClients{
Beam::Network::IpAddress{"127.0.0.1", 20000}, "username",
"password", Beam::Ref(socketThreadPool), Beam::Ref(timerThreadPool)};
serviceClients.Open();
auto& marketDataClient = serviceClients.GetMarketDataClient();

// Build the query.
auto query = Nexus::MarketDataService::SecurityMarketDataQuery();
query.SetIndex(Nexus::ParseSecurity("MSFT.NSDQ"));
query.SetRange(Beam::TimeService::ToUtcTime(
boost::posix_time::ptime{boost::gregorian::date{2016 , 8, 11}}),
Beam::TimeService::ToUtcTime(
boost::posix_time::ptime{boost::gregorian::date{2016, 8, 12}}));
query.SetSnapshotLimit(Beam::Queries::SnapshotLimit::Type::HEAD, 1000);

// Build the Queue to store the results.
auto queue = std::make_shared<Beam::Queue<Nexus::TimeAndSale>>();

// Submit the query to the market data service.
marketDataClient.QueryTimeAndSales(query, queue);

// Print the results
try {
while(true) {
std::cout << queue->Top().m_price << std::endl;
queue->Pop();
}
} catch(const Beam::PipeBrokenException&) {}
}
</syntaxhighlight>

===Python===
<syntaxhighlight lang=python line=line>
import beam
import nexus
import datetime

# Connect to the service.
service_clients = nexus.ApplicationServiceClients(
beam.network.IpAddress('127.0.0.1', 20000),
'username', 'password')

service_clients.open()
market_data_client = service_clients.get_market_data_client()

# Build the query.
query = nexus.market_data_service.SecurityMarketDataQuery()
query.index = nexus.parse_security('MSFT.NSDQ')
query.range = beam.queries.Range(beam.time_service.to_utc_time(
datetime.datetime(2016, 8, 11)), beam.time_service.to_utc_time(
datetime.datetime(2016, 8, 12)))
query.snapshot_limit = beam.queries.SnapshotLimit(
beam.queries.SnapshotLimit.Type.HEAD, 1000)

# Build the Queue to store the results.
queue = beam.Queue()

# Submit the query to the market data service.
market_data_client.query_time_and_sales(query, queue)

# Print the results
try:
while True:
print queue.top().price
queue.pop()
except:
pass
</syntaxhighlight>

Query

2018-01-21T22:22:35Z

Kamal: Created page with "Beam provides a series of classes to query historical and real time streaming data along a wide range of search criteria. It is one of the core APIs provided by Beam and used..."

Beam provides a series of classes to query historical and real time streaming data along a wide range of search criteria. It is one of the core APIs provided by Beam and used extensively in Spire to search through and disseminate market data and order executions.

==Structure==
Much of the data stored by Spire is in the form of a time series. The data is grouped together based on an index, for example time and sales data is indexed by its ticker symbol, order's are indexed by the account the order belongs to etc... For every item there is both a timestamp and a sequence number associated with it (to identify which data was published first in the event that two items have the same timestamp).

The most common type of query used in Spire to search through a time series is defined by the Beam::Queries::BasicQuery class. It consists of five components that are used to fully specify the type of data that is to be retrieved. Those five components are described in detail as follows:

===Index===
Every data item to query over must be associated with a single index. Specifying the value of the index to query over is defined by the class Beam::Queries::IndexedQuery.

===Range===
This component is defined by the class Beam::Queries::RangedIndex and it specifies the time range to search through as well as whether the query is strictly for historical data, real time data, or both. The class Beam::Queries::Range defines the semantics of a range and consists of a start value and an end value, both of which can be either a timestamp or a sequence number. Sequence numbers are defined by the class Beam::Queries::Sequence and it contains two special values which control the behavior of a query: LAST and PRESENT.

If the end of a range is specified to be PRESENT then the query will return all matching values that are currently stored in the database. If the end of a range is specified to be LAST or +infinity then in addition to returning all currently matching values, the query will also return matching values published in real time as they arrive.

Any other value used to mark the end of a range results in a strictly historical data query, even if the value denotes some point in the future.

===Snapshot limit===
This specifies the maximum number of historical data items to retrieve. This is typically used to prevent a query from returning too much data. The class Beam::Queries::SnapshotLimit contains two values, the size of the limit and the type of the limit. The size controls the maximum number of historical data items are returned by a query (with the special value of UNLIMITED if every value should be returned), as well as the type of snapshot. The type dictates whether items should be returned from the beginning of the series or the end of the series (the HEAD and TAIL respectively).

For example to query for the very last item currently stored in a time series, one can use a range of [FIRST, PRESENT] and a snapshot limit whose type is TAIL and whose size is 1. Alternatively to query for the first 10 values in a time series one would use a range of [FIRST, PRESENT] and a snapshot limit with type HEAD and size 10.

The class Beam::Queries::SnapshotLimitedQuery is the component used to indicate what the snapshot limit of a query should be. By default, this class specifies that no values are to be returned (its size is 0). This is because one is expected to be explicit about how much data is desired. The special value Beam::Queries::SnapshotLimit::Unlimited() can be used to explicitly indicate that there is no limit on how much data should be returned, however, it should be noted that the database engine handling the request may impose limits of its own regarding the maximum size of a snapshot.

===Interruption policy===
When querying for real time data it is possible for the query to get interrupted, usually due to a disconnection. This component, defined by Beam::Queries::InterruptableQuery, specifies what action should be taken in the event of such an interruption. The possible recovery options are defined by the enumerator Beam::Queries::InterruptionPolicy as follows:

RECOVER_DATA, the query should attempt to re-establish a connection to the database and recover any data that was published during the time that the query was interrupted.
IGNORE_CONTINUE, the query should attempt to re-establish a connection to the database but ignore any data that was published during the time that the query was interrupted. This is usually desirable when only live, real time data is desired as this option will omit any stale data published during the interruption.
BREAK_QUERY, the query should abort, indicating an exception. This is the default interruption policy.

===Filter===

==Example==
An example of a query for historical data might be to retrieve the first 1000 time and sales for MSFT.NSDQ from August 11th, 2016 to August 12th, 2016.

===C++===
#include <iostream>
#include "Nexus/MarketDataService/SecurityMarketDataQuery.hpp"
#include "Nexus/ServiceClients/ApplicationServiceClients.hpp"

int main() {
Beam::Network::SocketThreadPool socketThreadPool;
Beam::Threading::TimerThreadPool timerThreadPool;

// Connect to the service.
Nexus::ApplicationServiceClients serviceClients{
Beam::Network::IpAddress{"127.0.0.1", 20000}, "username",
"password", Beam::Ref(socketThreadPool), Beam::Ref(timerThreadPool)};
serviceClients.Open();
auto& marketDataClient = serviceClients.GetMarketDataClient();

// Build the query.
auto query = Nexus::MarketDataService::SecurityMarketDataQuery();
query.SetIndex(Nexus::ParseSecurity("MSFT.NSDQ"));
query.SetRange(Beam::TimeService::ToUtcTime(
boost::posix_time::ptime{boost::gregorian::date{2016 , 8, 11}}),
Beam::TimeService::ToUtcTime(
boost::posix_time::ptime{boost::gregorian::date{2016, 8, 12}}));
query.SetSnapshotLimit(Beam::Queries::SnapshotLimit::Type::HEAD, 1000);

// Build the Queue to store the results.
auto queue = std::make_shared<Beam::Queue<Nexus::TimeAndSale>>();

// Submit the query to the market data service.
marketDataClient.QueryTimeAndSales(query, queue);

// Print the results
try {
while(true) {
std::cout << queue->Top().m_price << std::endl;
queue->Pop();
}
} catch(const Beam::PipeBrokenException&) {}
}

===Python===
import beam
import nexus
import datetime

# Connect to the service.
service_clients = nexus.ApplicationServiceClients(
beam.network.IpAddress('127.0.0.1', 20000),
'username', 'password')

service_clients.open()
market_data_client = service_clients.get_market_data_client()

# Build the query.
query = nexus.market_data_service.SecurityMarketDataQuery()
query.index = nexus.parse_security('MSFT.NSDQ')
query.range = beam.queries.Range(beam.time_service.to_utc_time(
datetime.datetime(2016, 8, 11)), beam.time_service.to_utc_time(
datetime.datetime(2016, 8, 12)))
query.snapshot_limit = beam.queries.SnapshotLimit(
beam.queries.SnapshotLimit.Type.HEAD, 1000)

# Build the Queue to store the results.
queue = beam.Queue()

# Submit the query to the market data service.
market_data_client.query_time_and_sales(query, queue)

# Print the results
try:
while True:
print queue.top().price
queue.pop()
except:
pass