martino/marte_dev_tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Improved doc

Browse Source
This commit is contained in:
Martino Ferrari
2026-02-02 17:35:26 +01:00
parent 749eab0a32
commit ee9235c24d
5 changed files with 140 additions and 98 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
README.md
View File

@@ -16,9 +16,13 @@ Few additional features have been added to the standard MARTe configuration lang
- Multi file configuration support
- Multi file definition merging
- File level namespace / node
- Doc-strings support
- Pragmas for warning suppression / documentation
- File level namespace / node (`#package`)
- Variables and Constants
- Overrideable variables (`#var`)
- Fixed constants (`#let`)
- Powerful expressions (arithmetic, bitwise, string concatenation)
- Doc-strings support (`//#`) for objects, fields, and variables
- Pragmas (`//!`) for warning suppression / documentation
## Documentation

20
docs/CODE_DOCUMENTATION.md
View File

@@ -32,16 +32,17 @@ internal/
Responsible for converting MARTe configuration text into structured data.
* **Lexer (`lexer.go`)**: Tokenizes the input stream. Handles MARTe specific syntax like `#package`, `//!` pragmas, and `//#` docstrings. Supports standard identifiers and `#`-prefixed identifiers.
* **Parser (`parser.go`)**: Recursive descent parser. Converts tokens into a `Configuration` object containing definitions, comments, and pragmas.
* **AST (`ast.go`)**: Defines the node types (`ObjectNode`, `Field`, `Value`, `VariableDefinition`, etc.). All nodes implement the `Node` interface providing position information.
* **Lexer (`lexer.go`)**: Tokenizes the input stream. Handles MARTe specific syntax like `#package`, `#let`, `//!` pragmas, and `//#` docstrings. Supports standard identifiers and `#`-prefixed identifiers. Recognizes advanced number formats (hex `0x`, binary `0b`).
* **Parser (`parser.go`)**: Recursive descent parser. Converts tokens into a `Configuration` object containing definitions, comments, and pragmas. Implements expression parsing with precedence.
* **AST (`ast.go`)**: Defines the node types (`ObjectNode`, `Field`, `Value`, `VariableDefinition`, `BinaryExpression`, etc.). All nodes implement the `Node` interface providing position information.
### 2. `internal/index`
The brain of the system. It maintains a holistic view of the project.
* **ProjectTree**: The central data structure. It holds the root of the configuration hierarchy (`Root`), references, and isolated files.
* **ProjectNode**: Represents a logical node in the configuration. Since a node can be defined across multiple files (fragments), `ProjectNode` aggregates these fragments. It also stores locally defined variables in its `Variables` map.
* **ScanDirectory**: Recursively walks the project directory to find all `.marte` files, adding them to the tree even if they contain partial syntax errors.
* **ProjectNode**: Represents a logical node in the configuration. Since a node can be defined across multiple files (fragments), `ProjectNode` aggregates these fragments. It also stores locally defined variables and constants in its `Variables` map.
* **NodeMap**: A hash map index (`map[string][]*ProjectNode`) for $O(1)$ symbol lookups, optimizing `FindNode` operations.
* **Reference Resolution**: The `ResolveReferences` method links `Reference` objects to their target `ProjectNode` or `VariableDefinition`. It uses `ResolveName` (exported) which respects lexical scoping rules by searching the hierarchy upwards from the reference's container, using `FindNode` for deep searches within each scope.
@@ -53,10 +54,10 @@ Ensures configuration correctness.
* **Checks**:
* **Structure**: Duplicate fields, invalid content.
* **Schema**: Unifies nodes with CUE schemas (loaded via `internal/schema`) to validate types and mandatory fields.
* **Signals**: Verifies that signals referenced in GAMs exist in DataSources and match types.
* **Signals**: Verifies that signals referenced in GAMs exist in DataSources and match types. Performs project-wide consistency checks for implicit signals.
* **Threading**: Checks `CheckDataSourceThreading` to ensure non-multithreaded DataSources are not shared across threads in the same state.
* **Ordering**: `CheckINOUTOrdering` verifies that for `INOUT` signals, the producing GAM appears before the consuming GAM in the thread's execution list.
* **Variables**: `CheckVariables` validates variable values against their defined CUE types (e.g. `uint`, regex). `CheckUnresolvedVariables` ensures all used variables are defined.
* **Variables**: `CheckVariables` validates variable values against their defined CUE types. Prevents external overrides of `#let` constants. `CheckUnresolvedVariables` ensures all used variables are defined.
* **Unused**: Detects unused GAMs and Signals (suppressible via pragmas).
### 4. `internal/lsp`
@@ -64,11 +65,13 @@ Ensures configuration correctness.
Implements the Language Server Protocol.
* **Server (`server.go`)**: Handles JSON-RPC messages over stdio.
* **Evaluation**: Implements a lightweight expression evaluator to show evaluated values in Hover and completion snippets.
* **Incremental Sync**: Supports `textDocumentSync: 2`. `HandleDidChange` applies patches to the in-memory document buffers using `offsetAt` logic.
* **Features**:
* `HandleCompletion`: Context-aware suggestions (Schema fields, Signal references, Class names).
* `HandleHover`: Shows documentation, signal types, and usage analysis (e.g., "Used in GAMs: Controller (Input)").
* `HandleCompletion`: Context-aware suggestions (Macros, Schema fields, Signal references, Class names).
* `HandleHover`: Shows documentation (including docstrings for variables), evaluated signal types/dimensions, and usage analysis.
* `HandleDefinition` / `HandleReferences`: specific lookup using the `index`.
* `HandleRename`: Project-wide renaming supporting objects, fields, and signals (including implicit ones).
### 5. `internal/builder`
@@ -76,6 +79,7 @@ Merges multiple MARTe files into a single output.
* **Logic**: It parses all input files, builds a temporary `ProjectTree`, and then reconstructs the source code.
* **Merging**: It interleaves fields and subnodes from different file fragments to produce a coherent single-file configuration, respecting the `#package` hierarchy.
* **Evaluation**: Evaluates all expressions and variable references into concrete MARTe values in the final output. Prevents overrides of `#let` constants.
### 6. `internal/schema`

132
docs/CONFIGURATION_GUIDE.md
View File

@@ -20,7 +20,7 @@ Objects are defined using `+` (public/instantiated) or `$` (template/class-like)
### Fields and Values
- **Fields**: Alphanumeric identifiers (e.g., `Timeout`, `CycleTime`).
- **Values**:
- Integers: `10`, `-5`, `0xFA`
- Integers: `10`, `-5`, `0xFA`, `0b1011`
- Floats: `3.14`, `1e-3`
- Strings: `"Text"`
- Booleans: `true`, `false`
@@ -90,6 +90,28 @@ Common classes (`RealTimeApplication`, `StateMachine`, `IOGAM`, etc.) are built-
### Custom Schemas
You can extend the schema by creating a `.marte_schema.cue` file in your project root.
**Example: Adding a custom GAM**
```cue
package schema
#Classes: {
MyCustomGAM: {
// Metadata for Validator/LSP
#meta: {
direction: "INOUT" // "IN", "OUT", "INOUT"
multithreaded: false
}
// Fields
Gain: float
Offset?: float // Optional
InputSignals: {...}
OutputSignals: {...}
}
}
```
## 4. Variables and Constants
You can define variables to parameterize your configuration.
@@ -120,51 +142,38 @@ Constants are like variables but **cannot** be overridden externally. They are i
}
```
### Expressions
Variables and constants can be used in expressions:
- Arithmetic: `+`, `-`, `*`, `/`, `%`
- Bitwise: `&`, `|`, `^`
- String Concatenation: `..`
### Reference Syntax
Reference a variable or constant using `$` or `@`:
```marte
#var BasePath: string = "/tmp"
#let LogFile: string = @BasePath .. "/app.log"
Field = $MyVar
// or
Field = @MyVar
```
### Expressions
You can use operators in field values. Supported operators:
- **Math**: `+`, `-`, `*`, `/`, `%`, `^` (XOR), `&`, `|` (Bitwise)
- **String Concatenation**: `..`
- **Parentheses**: `(...)` for grouping
```marte
Field1 = 10 + 20 * 2 // 50
Field2 = "Hello " .. "World"
Field3 = ($MyVar + 5) * 2
```
### Build Override
You can override variable values during build (only for `#var`):
```bash
mdt build -vMyVar=200 src/*.marte
```
### Docstrings
Docstrings (`//#`) work for variables and constants and are displayed in the LSP hover information.
## 5. Pragmas
Macros can be controlled via pragmas:
- `//! allow(implicit)`: Suppress warnings for implicitly defined signals.
- `//! allow(unused)`: Suppress warnings for unused signals/GAMs.
- `//! ignore(not_consumed)`: Suppress ordering warnings for specific signals.
Pragmas can be global (top-level) or local to a node.
**Example: Adding a custom GAM**
```cue
package schema
#Classes: {
MyCustomGAM: {
// Metadata for Validator/LSP
#meta: {
direction: "INOUT" // "IN", "OUT", "INOUT"
multithreaded: false
}
// Fields
Gain: float
Offset?: float // Optional
InputSignals: {...}
OutputSignals: {...}
}
}
```
## 4. Multi-file Projects
## 5. Multi-file Projects
You can split your configuration into multiple files.
@@ -186,7 +195,7 @@ The `build` command merges all files.
mdt build -o final.marte src/*.marte
```
## 5. Pragmas (Suppressing Warnings)
## 6. Pragmas (Suppressing Warnings)
If validation is too strict, you can suppress warnings using pragmas (`//!`).
@@ -215,41 +224,11 @@ If validation is too strict, you can suppress warnings using pragmas (`//!`).
}
```
## 6. Variables
You can define variables using `#var`. The type expression supports CUE syntax.
```marte
#var MyVar: uint32 = 100
#var Env: "PROD" | "DEV" = "DEV"
```
### Usage
Reference a variable using `$` (preferred) or `@`:
```marte
Field = $MyVar
// or
Field = @MyVar
```
### Expressions
You can use operators in field values. Supported operators:
- **Math**: `+`, `-`, `*`, `/`, `%`, `^` (XOR), `&`, `|` (Bitwise)
- **String Concatenation**: `..`
```marte
Field1 = 10 + 20 * 2 // 50
Field2 = "Hello " .. "World"
Field3 = $MyVar + 5
```
### Build Override
You can override variable values during build:
```bash
mdt build -vMyVar=200 -vEnv="PROD" src/*.marte
```
- **Global Suppression**:
```marte
//! allow(unused)
//! allow(implicit)
```
## 7. Validation Rules (Detail)
@@ -267,5 +246,4 @@ To allow sharing, the DataSource class in the schema must have `#meta: multithre
### Implicit vs Explicit Signals
- **Explicit**: Signal defined in `DataSource.Signals`.
- **Implicit**: Signal used in GAM but not defined in DataSource. `mdt` reports a warning unless suppressed.
- **Consistency**: All references to the same logical signal (same name in same DataSource) must share the same `Type` and size properties.

41
docs/TUTORIAL.md
View File

@@ -148,7 +148,46 @@ make build
This produces `app.marte` (or `final_app.marte`), which contains the flattened, merged configuration ready for the MARTe framework.
## Step 6: Advanced - Custom Schema
## Step 6: Using Variables and Expressions
You can parameterize your application using variables. Let's define a constant for the sampling frequency.
Modify `src/app.marte`:
```marte
#package MyContollApp
//# Sampling frequency in Hz
#let SamplingFreq: uint32 = 100
+App = {
// ...
+Functions = {
+Converter = {
Class = IOGAM
InputSignals = {
TimeIn = {
DataSource = Timer
Type = uint32
Frequency = $SamplingFreq
Alias = Time
}
}
// ...
}
}
}
```
You can also use expressions for calculations:
```marte
#let CycleTime: float64 = 1.0 / $SamplingFreq
```
LSP hover will show you the evaluated values (e.g., `CycleTime: 0.01`).
## Step 7: Advanced - Custom Schema
Suppose you want to enforce that your DataSources support multithreading. You can modify `.marte_schema.cue`.

35
specification.md
View File

@@ -71,22 +71,31 @@ The LSP server should provide the following capabilities:
### Grammar
- `comment` : `//.*`
- `configuration`: `definition+`
- `configuration`: `(definition | macro)+`
- `definition`: `field = value | node = subnode`
- `macro`: `package | variable | constant`
- `field`: `[a-zA-Z][a-zA-Z0-9_\-]*`
- `node`: `[+$][a-zA-Z][a-zA-Z0-9_\-]*`
- `subnode`: `{ definition+ }`
- `value`: `string|int|float|bool|reference|array`
- `subnode`: `{ (definition | macro)+ }`
- `value`: `expression`
- `expression`: `atom | binary_expr | unary_expr`
- `atom`: `string | int | float | bool | reference | array | "(" expression ")"`
- `binary_expr`: `expression operator expression`
- `unary_expr`: `unary_operator expression`
- `operator`: `+ | - | * | / | % | & | | | ^ | ..`
- `unary_operator`: `- | !`
- `int`: `/-?[0-9]+|0b[01]+|0x[0-9a-fA-F]+`
- `float`: `-?[0-9]+\.[0-9]+|-?[0-9]+\.?[0-9]*e\-?[0-9]+`
- `float`: `-?[0-9]+\.[0-9]+|-?[0-9]+\.?[0-9]*[eE][+-]?[0-9]+`
- `bool`: `true|false`
- `string`: `".*"`
- `reference` : `string|.*`
- `array`: `{ value }`
- `reference` : `[a-zA-Z][a-zA-Z0-9_\-\.]* | @[a-zA-Z0-9_]+ | $[a-zA-Z0-9_]+`
- `array`: `{ (value | ",")* }`
#### Extended grammar
- `package` : `#package URI`
- `variable`: `#var NAME: TYPE [= expression]`
- `constant`: `#let NAME: TYPE = expression`
- `URI`: `PROJECT | PROJECT.PRJ_SUB_URI`
- `PRJ_SUB_URI`: `NODE | NODE.PRJ_SUB_URI`
- `docstring` : `//#.*`
@@ -97,13 +106,17 @@ The LSP server should provide the following capabilities:
- **Nodes (`+` / `$`)**: The prefixes `+` and `$` indicate that the node represents an object.
- **Constraint**: These nodes _must_ contain a field named `Class` within their subnode definition (across all files where the node is defined).
- **Signals**: Signals are considered nodes but **not** objects. They do not require a `Class` field.
- **Variables (`#var`)**: Define overrideable parameters. Can be overridden via CLI (`-vVAR=VAL`).
- **Constants (`#let`)**: Define fixed parameters. **Cannot** be overridden externally. Must have an initial value.
- **Expressions**: Evaluated during build and displayed evaluated in LSP hover documentation.
- **Docstrings (`//#`)**: Associated with the following definition (Node, Field, Variable, or Constant).
- **Pragmas (`//!`)**: Used to suppress specific diagnostics. The developer can use these to explain why a rule is being ignored. Supported pragmas:
- `//!unused: REASON` or `//!ignore(unused): REASON` - Suppress "Unused GAM" or "Unused Signal" warnings.
- `//!implicit: REASON` or `//!ignore(implicit): REASON` - Suppress "Implicitly Defined Signal" warnings.
- `//!allow(WARNING_TYPE): REASON` or `//!ignore(WARNING_TYPE): REASON` - Global suppression for a specific warning type across the whole project (supported: `unused`, `implicit`).
- `//!allow(WARNING_TYPE): REASON` or `//!ignore(WARNING_TYPE): REASON` - Global suppression for a specific warning type across the whole project (supported: `unused`, `implicit`, `not_consumed`, `not_produced`).
- `//!cast(DEF_TYPE, CUR_TYPE): REASON` - Suppress "Type Inconsistency" errors if types match.
- **Structure**: A configuration is composed by one or more definitions.
- **Strictness**: Any content that is not a valid comment (or pragma/docstring) or a valid definition (Field, Node, or Object) is **not allowed** and must generate a parsing error.
- **Structure**: A configuration is composed by one or more definitions or macros.
- **Strictness**: Any content that is not a valid comment (or pragma/docstring) or a valid definition/macro is **not allowed** and must generate a parsing error.
### Core MARTe Classes
@@ -124,6 +137,7 @@ MARTe configurations typically involve several main categories of objects:
- All signal definitions **must** include a `Type` field with a valid value.
- **Size Information**: Signals can optionally include `NumberOfDimensions` and `NumberOfElements` fields. If not explicitly defined, these default to `1`.
- **Property Matching**: Signal references in GAMs must match the properties (`Type`, `NumberOfElements`, `NumberOfDimensions`) of the defined signal in the `DataSource`.
- **Consistency**: Implicit signals used across different GAMs must share the same `Type` and size properties.
- **Extensibility**: Signal definitions can include additional fields as required by the specific application context.
- **Signal Reference Syntax**:
- Signals are referenced or defined in `InputSignals` or `OutputSignals` sub-nodes using one of the following formats:
@@ -145,6 +159,7 @@ MARTe configurations typically involve several main categories of objects:
```
In this case, `Alias` points to the DataSource signal name.
- **Implicit Definition Constraint**: If a signal is implicitly defined within a GAM, the `Type` field **must** be present in the reference block to define the signal's properties.
- **Renaming**: Renaming a signal (explicit or implicit) via LSP updates all its usages across all GAMs and DataSources in the project. Local aliases (`Alias = Name`) are preserved while their targets are updated.
- **Directionality**: DataSources and their signals are directional:
- `Input` (IN): Only providing data. Signals can only be used in `InputSignals`.
- `Output` (OUT): Only receiving data. Signals can only be used in `OutputSignals`.
@@ -155,9 +170,11 @@ MARTe configurations typically involve several main categories of objects:
The tool must build an index of the configuration to support LSP features and validations:
- **Recursive Indexing**: All `.marte` files in the project root and subdirectories are indexed automatically.
- **GAMs**: Referenced in `$APPLICATION.States.$STATE_NAME.Threads.$THREAD_NAME.Functions` (where `$APPLICATION` is a `RealTimeApplication` node).
- **Signals**: Referenced within the `InputSignals` and `OutputSignals` sub-nodes of a GAM.
- **DataSources**: Referenced within the `DataSource` field of a signal reference/definition.
- **Variables/Constants**: Referenced via `@NAME` or `$NAME` in expressions.
- **General References**: Objects can also be referenced in other fields (e.g., as targets for messages).
### Validation Rules