From ee9235c24d52e210e46081f632a67b8979d1ef8e Mon Sep 17 00:00:00 2001 From: Martino Ferrari Date: Mon, 2 Feb 2026 17:35:26 +0100 Subject: [PATCH] Improved doc --- README.md | 10 ++- docs/CODE_DOCUMENTATION.md | 20 +++--- docs/CONFIGURATION_GUIDE.md | 132 +++++++++++++++--------------------- docs/TUTORIAL.md | 41 ++++++++++- specification.md | 35 +++++++--- 5 files changed, 140 insertions(+), 98 deletions(-) diff --git a/README.md b/README.md index 9c6f5e3..b15aa90 100644 --- a/README.md +++ b/README.md @@ -16,9 +16,13 @@ Few additional features have been added to the standard MARTe configuration lang - Multi file configuration support - Multi file definition merging - - File level namespace / node -- Doc-strings support -- Pragmas for warning suppression / documentation + - File level namespace / node (`#package`) +- Variables and Constants + - Overrideable variables (`#var`) + - Fixed constants (`#let`) + - Powerful expressions (arithmetic, bitwise, string concatenation) +- Doc-strings support (`//#`) for objects, fields, and variables +- Pragmas (`//!`) for warning suppression / documentation ## Documentation diff --git a/docs/CODE_DOCUMENTATION.md b/docs/CODE_DOCUMENTATION.md index 66b60b8..4ca299a 100644 --- a/docs/CODE_DOCUMENTATION.md +++ b/docs/CODE_DOCUMENTATION.md @@ -32,16 +32,17 @@ internal/ Responsible for converting MARTe configuration text into structured data. -* **Lexer (`lexer.go`)**: Tokenizes the input stream. Handles MARTe specific syntax like `#package`, `//!` pragmas, and `//#` docstrings. Supports standard identifiers and `#`-prefixed identifiers. -* **Parser (`parser.go`)**: Recursive descent parser. Converts tokens into a `Configuration` object containing definitions, comments, and pragmas. -* **AST (`ast.go`)**: Defines the node types (`ObjectNode`, `Field`, `Value`, `VariableDefinition`, etc.). All nodes implement the `Node` interface providing position information. +* **Lexer (`lexer.go`)**: Tokenizes the input stream. Handles MARTe specific syntax like `#package`, `#let`, `//!` pragmas, and `//#` docstrings. Supports standard identifiers and `#`-prefixed identifiers. Recognizes advanced number formats (hex `0x`, binary `0b`). +* **Parser (`parser.go`)**: Recursive descent parser. Converts tokens into a `Configuration` object containing definitions, comments, and pragmas. Implements expression parsing with precedence. +* **AST (`ast.go`)**: Defines the node types (`ObjectNode`, `Field`, `Value`, `VariableDefinition`, `BinaryExpression`, etc.). All nodes implement the `Node` interface providing position information. ### 2. `internal/index` The brain of the system. It maintains a holistic view of the project. * **ProjectTree**: The central data structure. It holds the root of the configuration hierarchy (`Root`), references, and isolated files. -* **ProjectNode**: Represents a logical node in the configuration. Since a node can be defined across multiple files (fragments), `ProjectNode` aggregates these fragments. It also stores locally defined variables in its `Variables` map. +* **ScanDirectory**: Recursively walks the project directory to find all `.marte` files, adding them to the tree even if they contain partial syntax errors. +* **ProjectNode**: Represents a logical node in the configuration. Since a node can be defined across multiple files (fragments), `ProjectNode` aggregates these fragments. It also stores locally defined variables and constants in its `Variables` map. * **NodeMap**: A hash map index (`map[string][]*ProjectNode`) for $O(1)$ symbol lookups, optimizing `FindNode` operations. * **Reference Resolution**: The `ResolveReferences` method links `Reference` objects to their target `ProjectNode` or `VariableDefinition`. It uses `ResolveName` (exported) which respects lexical scoping rules by searching the hierarchy upwards from the reference's container, using `FindNode` for deep searches within each scope. @@ -53,10 +54,10 @@ Ensures configuration correctness. * **Checks**: * **Structure**: Duplicate fields, invalid content. * **Schema**: Unifies nodes with CUE schemas (loaded via `internal/schema`) to validate types and mandatory fields. - * **Signals**: Verifies that signals referenced in GAMs exist in DataSources and match types. + * **Signals**: Verifies that signals referenced in GAMs exist in DataSources and match types. Performs project-wide consistency checks for implicit signals. * **Threading**: Checks `CheckDataSourceThreading` to ensure non-multithreaded DataSources are not shared across threads in the same state. * **Ordering**: `CheckINOUTOrdering` verifies that for `INOUT` signals, the producing GAM appears before the consuming GAM in the thread's execution list. - * **Variables**: `CheckVariables` validates variable values against their defined CUE types (e.g. `uint`, regex). `CheckUnresolvedVariables` ensures all used variables are defined. + * **Variables**: `CheckVariables` validates variable values against their defined CUE types. Prevents external overrides of `#let` constants. `CheckUnresolvedVariables` ensures all used variables are defined. * **Unused**: Detects unused GAMs and Signals (suppressible via pragmas). ### 4. `internal/lsp` @@ -64,11 +65,13 @@ Ensures configuration correctness. Implements the Language Server Protocol. * **Server (`server.go`)**: Handles JSON-RPC messages over stdio. +* **Evaluation**: Implements a lightweight expression evaluator to show evaluated values in Hover and completion snippets. * **Incremental Sync**: Supports `textDocumentSync: 2`. `HandleDidChange` applies patches to the in-memory document buffers using `offsetAt` logic. * **Features**: - * `HandleCompletion`: Context-aware suggestions (Schema fields, Signal references, Class names). - * `HandleHover`: Shows documentation, signal types, and usage analysis (e.g., "Used in GAMs: Controller (Input)"). + * `HandleCompletion`: Context-aware suggestions (Macros, Schema fields, Signal references, Class names). + * `HandleHover`: Shows documentation (including docstrings for variables), evaluated signal types/dimensions, and usage analysis. * `HandleDefinition` / `HandleReferences`: specific lookup using the `index`. + * `HandleRename`: Project-wide renaming supporting objects, fields, and signals (including implicit ones). ### 5. `internal/builder` @@ -76,6 +79,7 @@ Merges multiple MARTe files into a single output. * **Logic**: It parses all input files, builds a temporary `ProjectTree`, and then reconstructs the source code. * **Merging**: It interleaves fields and subnodes from different file fragments to produce a coherent single-file configuration, respecting the `#package` hierarchy. +* **Evaluation**: Evaluates all expressions and variable references into concrete MARTe values in the final output. Prevents overrides of `#let` constants. ### 6. `internal/schema` diff --git a/docs/CONFIGURATION_GUIDE.md b/docs/CONFIGURATION_GUIDE.md index 4789fde..28109b2 100644 --- a/docs/CONFIGURATION_GUIDE.md +++ b/docs/CONFIGURATION_GUIDE.md @@ -20,7 +20,7 @@ Objects are defined using `+` (public/instantiated) or `$` (template/class-like) ### Fields and Values - **Fields**: Alphanumeric identifiers (e.g., `Timeout`, `CycleTime`). - **Values**: - - Integers: `10`, `-5`, `0xFA` + - Integers: `10`, `-5`, `0xFA`, `0b1011` - Floats: `3.14`, `1e-3` - Strings: `"Text"` - Booleans: `true`, `false` @@ -90,6 +90,28 @@ Common classes (`RealTimeApplication`, `StateMachine`, `IOGAM`, etc.) are built- ### Custom Schemas You can extend the schema by creating a `.marte_schema.cue` file in your project root. +**Example: Adding a custom GAM** + +```cue +package schema + +#Classes: { + MyCustomGAM: { + // Metadata for Validator/LSP + #meta: { + direction: "INOUT" // "IN", "OUT", "INOUT" + multithreaded: false + } + + // Fields + Gain: float + Offset?: float // Optional + InputSignals: {...} + OutputSignals: {...} + } +} +``` + ## 4. Variables and Constants You can define variables to parameterize your configuration. @@ -120,51 +142,38 @@ Constants are like variables but **cannot** be overridden externally. They are i } ``` -### Expressions -Variables and constants can be used in expressions: -- Arithmetic: `+`, `-`, `*`, `/`, `%` -- Bitwise: `&`, `|`, `^` -- String Concatenation: `..` +### Reference Syntax +Reference a variable or constant using `$` or `@`: ```marte -#var BasePath: string = "/tmp" -#let LogFile: string = @BasePath .. "/app.log" +Field = $MyVar +// or +Field = @MyVar +``` + +### Expressions +You can use operators in field values. Supported operators: +- **Math**: `+`, `-`, `*`, `/`, `%`, `^` (XOR), `&`, `|` (Bitwise) +- **String Concatenation**: `..` +- **Parentheses**: `(...)` for grouping + +```marte +Field1 = 10 + 20 * 2 // 50 +Field2 = "Hello " .. "World" +Field3 = ($MyVar + 5) * 2 +``` + +### Build Override +You can override variable values during build (only for `#var`): + +```bash +mdt build -vMyVar=200 src/*.marte ``` ### Docstrings Docstrings (`//#`) work for variables and constants and are displayed in the LSP hover information. -## 5. Pragmas -Macros can be controlled via pragmas: -- `//! allow(implicit)`: Suppress warnings for implicitly defined signals. -- `//! allow(unused)`: Suppress warnings for unused signals/GAMs. -- `//! ignore(not_consumed)`: Suppress ordering warnings for specific signals. - -Pragmas can be global (top-level) or local to a node. - -**Example: Adding a custom GAM** - -```cue -package schema - -#Classes: { - MyCustomGAM: { - // Metadata for Validator/LSP - #meta: { - direction: "INOUT" // "IN", "OUT", "INOUT" - multithreaded: false - } - - // Fields - Gain: float - Offset?: float // Optional - InputSignals: {...} - OutputSignals: {...} - } -} -``` - -## 4. Multi-file Projects +## 5. Multi-file Projects You can split your configuration into multiple files. @@ -186,7 +195,7 @@ The `build` command merges all files. mdt build -o final.marte src/*.marte ``` -## 5. Pragmas (Suppressing Warnings) +## 6. Pragmas (Suppressing Warnings) If validation is too strict, you can suppress warnings using pragmas (`//!`). @@ -215,41 +224,11 @@ If validation is too strict, you can suppress warnings using pragmas (`//!`). } ``` -## 6. Variables - -You can define variables using `#var`. The type expression supports CUE syntax. - -```marte -#var MyVar: uint32 = 100 -#var Env: "PROD" | "DEV" = "DEV" -``` - -### Usage -Reference a variable using `$` (preferred) or `@`: - -```marte -Field = $MyVar -// or -Field = @MyVar -``` - -### Expressions -You can use operators in field values. Supported operators: -- **Math**: `+`, `-`, `*`, `/`, `%`, `^` (XOR), `&`, `|` (Bitwise) -- **String Concatenation**: `..` - -```marte -Field1 = 10 + 20 * 2 // 50 -Field2 = "Hello " .. "World" -Field3 = $MyVar + 5 -``` - -### Build Override -You can override variable values during build: - -```bash -mdt build -vMyVar=200 -vEnv="PROD" src/*.marte -``` +- **Global Suppression**: + ```marte + //! allow(unused) + //! allow(implicit) + ``` ## 7. Validation Rules (Detail) @@ -267,5 +246,4 @@ To allow sharing, the DataSource class in the schema must have `#meta: multithre ### Implicit vs Explicit Signals - **Explicit**: Signal defined in `DataSource.Signals`. - **Implicit**: Signal used in GAM but not defined in DataSource. `mdt` reports a warning unless suppressed. - - +- **Consistency**: All references to the same logical signal (same name in same DataSource) must share the same `Type` and size properties. \ No newline at end of file diff --git a/docs/TUTORIAL.md b/docs/TUTORIAL.md index 1f761df..55aff81 100644 --- a/docs/TUTORIAL.md +++ b/docs/TUTORIAL.md @@ -148,7 +148,46 @@ make build This produces `app.marte` (or `final_app.marte`), which contains the flattened, merged configuration ready for the MARTe framework. -## Step 6: Advanced - Custom Schema +## Step 6: Using Variables and Expressions + +You can parameterize your application using variables. Let's define a constant for the sampling frequency. + +Modify `src/app.marte`: + +```marte +#package MyContollApp + +//# Sampling frequency in Hz +#let SamplingFreq: uint32 = 100 + ++App = { + // ... + +Functions = { + +Converter = { + Class = IOGAM + InputSignals = { + TimeIn = { + DataSource = Timer + Type = uint32 + Frequency = $SamplingFreq + Alias = Time + } + } + // ... + } + } +} +``` + +You can also use expressions for calculations: + +```marte +#let CycleTime: float64 = 1.0 / $SamplingFreq +``` + +LSP hover will show you the evaluated values (e.g., `CycleTime: 0.01`). + +## Step 7: Advanced - Custom Schema Suppose you want to enforce that your DataSources support multithreading. You can modify `.marte_schema.cue`. diff --git a/specification.md b/specification.md index f6a1dc8..c93ebec 100644 --- a/specification.md +++ b/specification.md @@ -71,22 +71,31 @@ The LSP server should provide the following capabilities: ### Grammar - `comment` : `//.*` -- `configuration`: `definition+` +- `configuration`: `(definition | macro)+` - `definition`: `field = value | node = subnode` +- `macro`: `package | variable | constant` - `field`: `[a-zA-Z][a-zA-Z0-9_\-]*` - `node`: `[+$][a-zA-Z][a-zA-Z0-9_\-]*` -- `subnode`: `{ definition+ }` -- `value`: `string|int|float|bool|reference|array` +- `subnode`: `{ (definition | macro)+ }` +- `value`: `expression` +- `expression`: `atom | binary_expr | unary_expr` +- `atom`: `string | int | float | bool | reference | array | "(" expression ")"` +- `binary_expr`: `expression operator expression` +- `unary_expr`: `unary_operator expression` +- `operator`: `+ | - | * | / | % | & | | | ^ | ..` +- `unary_operator`: `- | !` - `int`: `/-?[0-9]+|0b[01]+|0x[0-9a-fA-F]+` -- `float`: `-?[0-9]+\.[0-9]+|-?[0-9]+\.?[0-9]*e\-?[0-9]+` +- `float`: `-?[0-9]+\.[0-9]+|-?[0-9]+\.?[0-9]*[eE][+-]?[0-9]+` - `bool`: `true|false` - `string`: `".*"` -- `reference` : `string|.*` -- `array`: `{ value }` +- `reference` : `[a-zA-Z][a-zA-Z0-9_\-\.]* | @[a-zA-Z0-9_]+ | $[a-zA-Z0-9_]+` +- `array`: `{ (value | ",")* }` #### Extended grammar - `package` : `#package URI` +- `variable`: `#var NAME: TYPE [= expression]` +- `constant`: `#let NAME: TYPE = expression` - `URI`: `PROJECT | PROJECT.PRJ_SUB_URI` - `PRJ_SUB_URI`: `NODE | NODE.PRJ_SUB_URI` - `docstring` : `//#.*` @@ -97,13 +106,17 @@ The LSP server should provide the following capabilities: - **Nodes (`+` / `$`)**: The prefixes `+` and `$` indicate that the node represents an object. - **Constraint**: These nodes _must_ contain a field named `Class` within their subnode definition (across all files where the node is defined). - **Signals**: Signals are considered nodes but **not** objects. They do not require a `Class` field. +- **Variables (`#var`)**: Define overrideable parameters. Can be overridden via CLI (`-vVAR=VAL`). +- **Constants (`#let`)**: Define fixed parameters. **Cannot** be overridden externally. Must have an initial value. +- **Expressions**: Evaluated during build and displayed evaluated in LSP hover documentation. +- **Docstrings (`//#`)**: Associated with the following definition (Node, Field, Variable, or Constant). - **Pragmas (`//!`)**: Used to suppress specific diagnostics. The developer can use these to explain why a rule is being ignored. Supported pragmas: - `//!unused: REASON` or `//!ignore(unused): REASON` - Suppress "Unused GAM" or "Unused Signal" warnings. - `//!implicit: REASON` or `//!ignore(implicit): REASON` - Suppress "Implicitly Defined Signal" warnings. - - `//!allow(WARNING_TYPE): REASON` or `//!ignore(WARNING_TYPE): REASON` - Global suppression for a specific warning type across the whole project (supported: `unused`, `implicit`). + - `//!allow(WARNING_TYPE): REASON` or `//!ignore(WARNING_TYPE): REASON` - Global suppression for a specific warning type across the whole project (supported: `unused`, `implicit`, `not_consumed`, `not_produced`). - `//!cast(DEF_TYPE, CUR_TYPE): REASON` - Suppress "Type Inconsistency" errors if types match. -- **Structure**: A configuration is composed by one or more definitions. -- **Strictness**: Any content that is not a valid comment (or pragma/docstring) or a valid definition (Field, Node, or Object) is **not allowed** and must generate a parsing error. +- **Structure**: A configuration is composed by one or more definitions or macros. +- **Strictness**: Any content that is not a valid comment (or pragma/docstring) or a valid definition/macro is **not allowed** and must generate a parsing error. ### Core MARTe Classes @@ -124,6 +137,7 @@ MARTe configurations typically involve several main categories of objects: - All signal definitions **must** include a `Type` field with a valid value. - **Size Information**: Signals can optionally include `NumberOfDimensions` and `NumberOfElements` fields. If not explicitly defined, these default to `1`. - **Property Matching**: Signal references in GAMs must match the properties (`Type`, `NumberOfElements`, `NumberOfDimensions`) of the defined signal in the `DataSource`. + - **Consistency**: Implicit signals used across different GAMs must share the same `Type` and size properties. - **Extensibility**: Signal definitions can include additional fields as required by the specific application context. - **Signal Reference Syntax**: - Signals are referenced or defined in `InputSignals` or `OutputSignals` sub-nodes using one of the following formats: @@ -145,6 +159,7 @@ MARTe configurations typically involve several main categories of objects: ``` In this case, `Alias` points to the DataSource signal name. - **Implicit Definition Constraint**: If a signal is implicitly defined within a GAM, the `Type` field **must** be present in the reference block to define the signal's properties. +- **Renaming**: Renaming a signal (explicit or implicit) via LSP updates all its usages across all GAMs and DataSources in the project. Local aliases (`Alias = Name`) are preserved while their targets are updated. - **Directionality**: DataSources and their signals are directional: - `Input` (IN): Only providing data. Signals can only be used in `InputSignals`. - `Output` (OUT): Only receiving data. Signals can only be used in `OutputSignals`. @@ -155,9 +170,11 @@ MARTe configurations typically involve several main categories of objects: The tool must build an index of the configuration to support LSP features and validations: +- **Recursive Indexing**: All `.marte` files in the project root and subdirectories are indexed automatically. - **GAMs**: Referenced in `$APPLICATION.States.$STATE_NAME.Threads.$THREAD_NAME.Functions` (where `$APPLICATION` is a `RealTimeApplication` node). - **Signals**: Referenced within the `InputSignals` and `OutputSignals` sub-nodes of a GAM. - **DataSources**: Referenced within the `DataSource` field of a signal reference/definition. +- **Variables/Constants**: Referenced via `@NAME` or `$NAME` in expressions. - **General References**: Objects can also be referenced in other fields (e.g., as targets for messages). ### Validation Rules