martino/marte_dev_tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Improving CLI tool and improving documentation

Browse Source
This commit is contained in:
Martino Ferrari
2026-01-28 13:32:32 +01:00
parent 31996ae710
commit 01bcd66594
15 changed files with 895 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files

106
docs/CODE_DOCUMENTATION.md Normal file
View File

@@ -0,0 +1,106 @@
# mdt Internal Code Documentation
This document provides a detailed overview of the `mdt` codebase architecture and internal components.
## Architecture Overview
`mdt` is built as a modular system where core functionalities are separated into internal packages. The data flow typically follows this pattern:
1. **Parsing**: Source code is parsed into an Abstract Syntax Tree (AST).
2. **Indexing**: ASTs from multiple files are aggregated into a unified `ProjectTree`.
3. **Processing**: The `ProjectTree` is used by the Validator, Builder, and LSP server to perform their respective tasks.
## Package Structure
```
cmd/
mdt/ # Application entry point (CLI)
internal/
builder/ # Logic for merging and building configurations
formatter/ # Code formatting engine
index/ # Symbol table and project structure management
logger/ # Centralized logging
lsp/ # Language Server Protocol implementation
parser/ # Lexer, Parser, and AST definitions
schema/ # CUE schema loading and integration
validator/ # Semantic analysis and validation logic
```
## Core Packages
### 1. `internal/parser`
Responsible for converting MARTe configuration text into structured data.
* **Lexer (`lexer.go`)**: Tokenizes the input stream. Handles MARTe specific syntax like `#package`, `//!` pragmas, and `//#` docstrings. Supports standard identifiers and `#`-prefixed identifiers.
* **Parser (`parser.go`)**: Recursive descent parser. Converts tokens into a `Configuration` object containing definitions, comments, and pragmas.
* **AST (`ast.go`)**: Defines the node types (`ObjectNode`, `Field`, `Value`, etc.). All nodes implement the `Node` interface providing position information.
### 2. `internal/index`
The brain of the system. It maintains a holistic view of the project.
* **ProjectTree**: The central data structure. It holds the root of the configuration hierarchy (`Root`), references, and isolated files.
* **ProjectNode**: Represents a logical node in the configuration. Since a node can be defined across multiple files (fragments), `ProjectNode` aggregates these fragments.
* **NodeMap**: A hash map index (`map[string][]*ProjectNode`) for $O(1)$ symbol lookups, optimizing `FindNode` operations.
* **Reference Resolution**: The `ResolveReferences` method links `Reference` objects to their target `ProjectNode` using the `NodeMap`.
### 3. `internal/validator`
Ensures configuration correctness.
* **Validator**: Iterates over the `ProjectTree` to check rules.
* **Checks**:
* **Structure**: Duplicate fields, invalid content.
* **Schema**: Unifies nodes with CUE schemas (loaded via `internal/schema`) to validate types and mandatory fields.
* **Signals**: Verifies that signals referenced in GAMs exist in DataSources and match types.
* **Threading**: Checks `checkDataSourceThreading` to ensure non-multithreaded DataSources are not shared across threads in the same state.
* **Unused**: Detects unused GAMs and Signals (suppressible via pragmas).
### 4. `internal/lsp`
Implements the Language Server Protocol.
* **Server (`server.go`)**: Handles JSON-RPC messages over stdio.
* **Incremental Sync**: Supports `textDocumentSync: 2`. `HandleDidChange` applies patches to the in-memory document buffers using `offsetAt` logic.
* **Features**:
* `HandleCompletion`: Context-aware suggestions (Schema fields, Signal references, Class names).
* `HandleHover`: Shows documentation, signal types, and usage analysis (e.g., "Used in GAMs: Controller (Input)").
* `HandleDefinition` / `HandleReferences`: specific lookup using the `index`.
### 5. `internal/builder`
Merges multiple MARTe files into a single output.
* **Logic**: It parses all input files, builds a temporary `ProjectTree`, and then reconstructs the source code.
* **Merging**: It interleaves fields and subnodes from different file fragments to produce a coherent single-file configuration, respecting the `#package` hierarchy.
### 6. `internal/schema`
Manages CUE schemas.
* **Loading**: Loads the embedded default schema (`marte.cue`) and merges it with any user-provided `.marte_schema.cue`.
* **Metadata**: Handles the `#meta` field in schemas to extract properties like `direction` and `multithreaded` support for the validator.
## Key Data Flows
### Reference Resolution
1. **Scan**: Files are parsed and added to the `ProjectTree`.
2. **Index**: `RebuildIndex` populates `NodeMap`.
3. **Resolve**: `ResolveReferences` iterates all recorded references (values) and calls `FindNode`.
4. **Link**: If found, `ref.Target` is set to the `ProjectNode`.
### Validation Lifecycle
1. `mdt check` or LSP `didChange` triggers validation.
2. A new `Validator` is created with the current `Tree`.
3. `ValidateProject` is called.
4. It walks the tree, runs checks, and populates `Diagnostics`.
5. Diagnostics are printed (CLI) or published via `textDocument/publishDiagnostics` (LSP).
### Threading Check Logic
1. Finds the `RealTimeApplication` node.
2. Iterates through `States` and `Threads`.
3. For each Thread, resolves the `Functions` (GAMs).
4. For each GAM, resolves connected `DataSources` via Input/Output signals.
5. Maps `DataSource -> Thread` within the context of a State.
6. If a DataSource is seen in >1 Thread, it checks the `#meta.multithreaded` property. If false (default), an error is raised.

162
docs/CONFIGURATION_GUIDE.md Normal file
View File

@@ -0,0 +1,162 @@
# MARTe Configuration Guide
This guide explains the syntax, features, and best practices for writing MARTe configurations using `mdt`.
## 1. Syntax Overview
MARTe configurations use a hierarchical object-oriented syntax.
### Objects (Nodes)
Objects are defined using `+` (public/instantiated) or `$` (template/class-like) prefixes. Every object **must** have a `Class` field.
```marte
+MyObject = {
Class = MyClass
Field1 = 100
Field2 = "Hello"
}
```
### Fields and Values
- **Fields**: Alphanumeric identifiers (e.g., `Timeout`, `CycleTime`).
- **Values**:
- Integers: `10`, `-5`, `0xFA`
- Floats: `3.14`, `1e-3`
- Strings: `"Text"`
- Booleans: `true`, `false`
- References: `MyObject`, `MyObject.SubNode`
- Arrays: `{ 1 2 3 }` or `{ "A" "B" }`
### Comments and Documentation
- Line comments: `// This is a comment`
- Docstrings: `//# This documents the following node`. These appear in hover tooltips.
```marte
//# This is the main application
+App = { ... }
```
## 2. Signals and Data Flow
Signals define how data moves between DataSources (drivers) and GAMs (algorithms).
### Defining Signals
Signals are typically defined in a `DataSource`. They must have a `Type`.
```marte
+MyDataSource = {
Class = GAMDataSource
Signals = {
Signal1 = { Type = uint32 }
Signal2 = { Type = float32 }
}
}
```
### Using Signals in GAMs
GAMs declare inputs and outputs. You can refer to signals directly or alias them.
```marte
+MyGAM = {
Class = IOGAM
InputSignals = {
Signal1 = {
DataSource = MyDataSource
Type = uint32 // Must match DataSource definition
}
MyAlias = {
Alias = Signal2
DataSource = MyDataSource
Type = float32
}
}
}
```
### Threading Rules
**Validation Rule**: A DataSource that is **not** marked as multithreaded (default) cannot be used by GAMs running in different threads within the same State.
To allow sharing, the DataSource class in the schema must have `#meta: multithreaded: true`.
## 3. Schemas and Validation
`mdt` validates your configuration against CUE schemas.
### Built-in Schema
Common classes (`RealTimeApplication`, `StateMachine`, `IOGAM`, etc.) are built-in.
### Custom Schemas
You can extend the schema by creating a `.marte_schema.cue` file in your project root.
**Example: Adding a custom GAM**
```cue
package schema
#Classes: {
MyCustomGAM: {
// Metadata for Validator/LSP
#meta: {
direction: "INOUT" // "IN", "OUT", "INOUT"
multithreaded: false
}
// Fields
Gain: float
Offset?: float // Optional
InputSignals: {...}
OutputSignals: {...}
}
}
```
## 4. Multi-file Projects
You can split your configuration into multiple files.
### Namespaces
Use `#package` to define where the file's content fits in the hierarchy.
**file1.marte**
```marte
#package MyApp.Controller
+MyController = { ... }
```
This places `MyController` under `MyApp.Controller`.
### Building
The `build` command merges all files.
```bash
mdt build -o final.marte src/*.marte
```
## 5. Pragmas (Suppressing Warnings)
If validation is too strict, you can suppress warnings using pragmas (`//!`).
- **Suppress Unused Warning**:
```marte
+MyGAM = {
Class = IOGAM
//! ignore(unused): This GAM is triggered externally
}
```
- **Suppress Implicit Signal Warning**:
```marte
InputSignals = {
//! ignore(implicit)
ImplicitSig = { Type = uint32 }
}
```
- **Type Casting**:
```marte
Sig1 = {
//! cast(uint32, int32): Intentional mismatch
DataSource = DS
Type = int32
}
```

158
docs/EDITOR_INTEGRATION.md Normal file
View File

@@ -0,0 +1,158 @@
# Editor Integration Guide
`mdt` includes a Language Server Protocol (LSP) implementation that provides features like:
- Syntax highlighting and error reporting
- Auto-completion
- Go to Definition / References
- Hover documentation
- Symbol renaming
The LSP server is started via the command:
```bash
mdt lsp
```
It communicates via **stdio**.
## VS Code
You can use a generic LSP extension like [Generic LSP Client](https://marketplace.visualstudio.com/items?itemName=summne.vscode-generic-lsp-client) or configure a custom task.
**Using "Run on Save" or similar extensions is an option, but for true LSP support:**
1. Install the **"glspc"** (Generic LSP Client) extension or similar.
2. Configure it in your `settings.json`:
```json
"glspc.languageServer configurations": [
{
"languageId": "marte",
"command": "mdt",
"args": ["lsp"],
"rootUri": "${workspaceFolder}"
}
]
```
3. Associate `.marte` files with the language ID:
```json
"files.associations": {
"*.marte": "marte"
}
```
## Neovim (Native LSP)
Add the following to your `init.lua` or `init.vim` (using `nvim-lspconfig`):
```lua
local lspconfig = require'lspconfig'
local configs = require'lspconfig.configs'
if not configs.marte then
configs.marte = {
default_config = {
cmd = {'mdt', 'lsp'},
filetypes = {'marte'},
root_dir = lspconfig.util.root_pattern('.git', 'go.mod', '.marte_schema.cue'),
settings = {},
},
}
end
lspconfig.marte.setup{}
-- Add filetype detection
vim.cmd([[
autocmd BufNewFile,BufRead *.marte setfiletype marte
]])
```
## Helix
Add this to your `languages.toml` (usually in `~/.config/helix/languages.toml`):
```toml
[[language]]
name = "marte"
scope = "source.marte"
injection-regex = "marte"
file-types = ["marte"]
roots = [".git", ".marte_schema.cue"]
comment-token = "//"
indent = { tab-width = 2, unit = " " }
language-servers = [ "mdt-lsp" ]
[language-server.mdt-lsp]
command = "mdt"
args = ["lsp"]
```
## Vim
### Using `vim-lsp`
```vim
if executable('mdt')
au User lsp_setup call lsp#register_server({
\ 'name': 'mdt-lsp',
\ 'cmd': {server_info->['mdt', 'lsp']},
\ 'whitelist': ['marte'],
\ })
endif
au BufRead,BufNewFile *.marte set filetype=marte
```
### Using `ALE`
```vim
call ale#linter#define('marte', {
\ 'name': 'mdt',
\ 'lsp': 'stdio',
\ 'executable': 'mdt',
\ 'command': '%e lsp',
\ 'project_root': function('ale#handlers#python#FindProjectRoot'),
\})
```
## Zed
Add to your `settings.json`:
```json
"lsp": {
"marte": {
"binary": {
"path": "mdt",
"arguments": ["lsp"]
}
}
}
```
## Kakoune (kak-lsp)
In your `kak-lsp.toml`:
```toml
[language.marte]
filetypes = ["marte"]
roots = [".git", ".marte_schema.cue"]
command = "mdt"
args = ["lsp"]
```
## Eclipse
1. Install **LSP4E** plugin.
2. Go to **Preferences > Language Servers**.
3. Add a new Language Server:
- **Content Type**: Text / Custom (Associate `*.marte` with a content type).
- **Launch configuration**: Program.
- **Command**: `mdt`
- **Arguments**: `lsp`
- **Input/Output**: Standard Input/Output.

173
docs/TUTORIAL.md Normal file
View File

@@ -0,0 +1,173 @@
# Creating a MARTe Application with mdt
This tutorial will guide you through creating, building, and validating a complete MARTe application using the `mdt` toolset.
## Prerequisites
- `mdt` installed and available in your PATH.
- `make` (optional but recommended).
## Step 1: Initialize the Project
Start by creating a new project named `MyControlApp`.
```bash
mdt init MyControlApp
cd MyControlApp
```
This command creates a standard project structure:
- `Makefile`: For building and checking the project.
- `.marte_schema.cue`: For defining custom schemas (if needed).
- `src/app.marte`: The main application definition.
- `src/components.marte`: A placeholder for defining components (DataSources).
## Step 2: Define Components
Open `src/components.marte`. This file uses the `#package App.Data` namespace, meaning all definitions here will be children of `App.Data`.
Let's define a **Timer** (input source) and a **Logger** (output destination).
```marte
#package MyContollApp.App.Data
+DDB = {
Class = GAMDataSource
}
+TimingDataSource = {
Class = TimingDataSource
}
+Timer = {
Class = LinuxTimer
Signals = {
Counter = {
Type = uint32
}
Time = {
Type = uint32
}
}
}
+Logger = {
Class = LoggerDataSource
Signals = {
LogValue = {
Type = float32
}
}
}
```
## Step 3: Implement Logic (GAM)
Open `src/app.marte`. This file defines the `App` node.
We will add a GAM that takes the time from the Timer, converts it, and logs it.
Add the GAM definition inside the `+Main` object (or as a separate object if you prefer modularity). Let's modify `src/app.marte`:
```marte
#package MyContollApp
+App = {
Class = RealTimeApplication
+Functions = {
Class = RefenceContainer
// Define the GAM
+Converter = {
Class = IOGAM
InputSignals = {
TimeIn = {
DataSource = Timer
Type = uint32
Frequency = 100 //Hz
Alias = Time // Refers to 'Time' signal in Timer
}
}
OutputSignals = {
LogOut = {
DataSource = Logger
Type = float32
Alias = LogValue
}
}
}
}
+States = {
Class = ReferenceContainer
+Run = {
Class = RealTimeState
+MainThread = {
Class = RealTimeThread
Functions = { Converter } // Run our GAM
}
}
}
+Data = {
Class = ReferenceContainer
DefaultDataSource = DDB
}
+Scheduler = {
Class = GAMScheduler
TimingDataSource = TimingDataSource
}
}
```
## Step 4: Validate
Run the validation check to ensure everything is correct (types match, references are valid).
```bash
mdt check src/*.marte
```
Or using Make:
```bash
make check
```
If you made a mistake (e.g., mismatched types), `mdt` will report an error.
## Step 5: Build
Merge all files into a single configuration file.
```bash
mdt build -o final_app.marte src/*.marte
```
Or using Make:
```bash
make build
```
This produces `app.marte` (or `final_app.marte`), which contains the flattened, merged configuration ready for the MARTe framework.
## Step 6: Advanced - Custom Schema
Suppose you want to enforce that your DataSources support multithreading. You can modify `.marte_schema.cue`.
```cue
package schema
#Classes: {
// Enforce that LinuxTimer must be multithreaded (example)
LinuxTimer: {
#meta: {
multithreaded: true
}
...
}
}
```
Now, if you use `LinuxTimer` in multiple threads, `mdt check` will allow it (because of `#meta.multithreaded: true`). By default, it would disallow it.
## Conclusion
You have successfully initialized, implemented, validated, and built a MARTe application using `mdt`.