tick-collection-ops
Operate and troubleshoot the MT5 tick collection system. HOST MOVED 2026-08-07 - MT5 now runs on the FXView VPS (Windows scheduled tasks, C:\odb-tickrings), NOT <gpu-host>/Linux-Wine/systemd. EA, ring format and gap-detection reasoning unchanged. Covers gap detection, restart recovery, daily ops.
How do I install this agent skill?
npx skills add https://github.com/terrylica/cc-skills --skill tick-collection-opsIs this agent skill safe to install?
- Gen Agent Trust Hubwarn
The skill provides operational guidance for a financial data collection system but introduces security risks through self-modification directives and recommendations for obfuscated command execution. It instructs the AI to rewrite its own instructions based on runtime observations, which could be exploited via indirect prompt injection. Additionally, it promotes the use of base64-encoded PowerShell commands, a common technique for bypassing security monitoring and hiding command intent.
- Socketpass
No alerts
- Snykpass
Risk: LOW · No issues
What does this agent skill do?
MT5 Tick Collection Operations
Self-Evolving Skill: This skill improves through use. If instructions are wrong, parameters drifted, or a workaround was needed — fix this file immediately, don't defer. Only update for real, reproducible issues.
⚠️ HOST SUPERSEDED — 2026-08-07
MetaTrader 5 no longer runs on <gpu-host>. It runs exclusively on a broker VPS (Tailscale host
mt5-host,<tailscale-ip>, Windows), on a demo account. The Mac-local Wine bench was deleted 2026-08-06; <gpu-host>'s Wine terminal + itsxvfb/xfce/x11vncdesktop andfxview-sidecarare being decommissioned.On the VPS the topology is Windows scheduled tasks, not Linux systemd:
Concern <gpu-host> (retired) Broker VPS (current) Terminal mt5.serviceunder Wine,DISPLAY=:99terminal64.exe, native Windows, real sessionTick rings /dev/shm/tick_ring_<SYM>C:\odb-tickrings\tick_ring_<SYM>Ring → bus — odb-tapegateway.exe→ taskOdbTickRingToNatsTapeGatewayBus — nats-server.exe:4222→ taskOdbTapeBusNatsJetStreamServerOrders — order-service :50060→ taskODB-MT5-OrderServiceSymbols EUR/GBP/XAU/XAG EUR/GBP/XAU only (silver excluded per directive 2026-05-26) Service control systemctlGet-ScheduledTask/Start-ScheduledTaskReach it with
ssh <mt5-host>. PowerShell over SSH eats$_under shell quoting — pass commands base64-encoded viapowershell -NoProfile -EncodedCommand <base64-utf16le>.Everything below still describes the Linux/Wine deployment. It is retained because the EA, the
tick_writer.dllring format and the gap-detection reasoning are unchanged and still authoritative — only the HOST and the service manager moved. Do not use it to stand up a new Linux MT5.Canonical:
<research-repo>/docs/adr/<date>-retire-<host>-as-a-metatrader-host.mdanddocs/decision-register-<topic>.md.
Operate, monitor, and troubleshoot the zero-gap tick collection system running on Linux via Wine.
When to Use This Skill
Use this skill when:
- Deploying or restarting tick collection on <gpu-host>
- Diagnosing tick gaps or missing data
- Troubleshooting Wine or MT5 issues
- Understanding systemd service topology
- Investigating crash recovery files
- Checking tick collection health
System Architecture
TickCollector EA (MQL5) --> tick_writer.dll (Rust) --> Parquet files
OnTick() callback C-ABI FFI ZSTD compressed
CopyTicksRange() Buffered writes One file per day
Watermark dedup Row group flush
- TickCollector EA runs on MT5 via Wine on headless Linux (<gpu-host>)
- EA captures every tick via OnTick() callback + OnTimer() fallback (1-second interval)
- Ticks buffered in EA, flushed to Rust DLL (tick_writer.dll) via C-ABI FFI
- Rust DLL writes Parquet files with ZSTD level 3 compression
- One EA instance per symbol per chart -- this is a CRITICAL constraint (single symbol per EA)
Production Topology
| Resource | Path |
|---|---|
| Git repo | ~/eon/mql5/ |
| Wine prefix | ~/.mt5/ |
| MT5 install | ~/.mt5/drive_c/Program Files/MetaTrader 5/ |
| EA source | {repo}/mql5_ea/TickCollector.mq5 |
| DLL (Windows) | {mt5}/MQL5/Libraries/tick_writer.dll |
| Tick data | {mt5}/tick_data/ (symlinked to ODB cache) |
| Systemd units | systemd user units: xvfb, xfce, x11vnc, mt5 |
Systemd Service Chain
xvfb (virtual framebuffer)
--> xfce (desktop environment)
--> x11vnc (VNC access)
--> mt5 (MetaTrader 5 via Wine)
All are user-level systemd units. MT5 depends on the display stack. The chain must start in order -- xvfb first, mt5 last.
Daily File Rotation
- New Parquet file created each trading day
- Filename pattern:
{SYMBOL}_{YYYYMMDD}.parquet - EA detects day change and calls
tw_close(finalize current file) +tw_init(open new file) - Trading day runs Sunday open to Monday open (forex hours)
Crash Recovery
On EA restart:
- EA calls
tw_get_last_time_mscto read resume watermark from the last Parquet file footer - Watermark is the timestamp of the last tick written -- resume from there
- If an existing file is found for today, new file gets
_1,_2suffix (not overwrite) - All recovery segments are valid, complete Parquet files with footers
- Resume watermark prevents duplicate ticks after restart
- Gap between shutdown and restart is backfilled via
CopyTicksRangefrom the watermark
The Rust DLL renames corrupt/incomplete files to _partial suffix rather than attempting recovery.
Gap Detection
Use DuckDB to check for gaps between consecutive ticks:
WITH ticks AS (
SELECT time_msc,
lead(time_msc) OVER (ORDER BY time_msc) - time_msc AS gap_ms
FROM read_parquet('{base_path}/FXVIEW_{SYMBOL}/{YYYY}/{SYMBOL}_{YYYYMMDD}.parquet')
)
SELECT time_msc, gap_ms,
make_timestamp(time_msc * 1000) AS ts
FROM ticks
WHERE gap_ms > 60000 -- gaps > 1 minute
ORDER BY gap_ms DESC;
Interpreting results:
- Weekend gaps are normal (Friday close to Sunday open)
- Mid-session gaps indicate collection interruption
- Gaps < 1 minute are normal during low-activity periods
- Check systemd journal for correlating restart events
Wine Gotchas
These are CRITICAL anti-patterns -- violations cause silent crashes or data loss:
- NEVER use
%I64uor%I64dformat specifiers in MQL5 -- crashes Wine silently. UseIntegerToString()instead. - Wine cannot follow symlinks for file access -- use physical file copies for EA and DLL deployment
- Headless Linux requires Xvfb virtual framebuffer -- MT5 GUI needs a display even when no monitor is connected
- compile.sh uses Wine CLI for compilation -- no MetaEditor GUI needed
- MetaEditor returns exit code 1 even on success under Wine -- check compile log instead
- Compile log is UTF-16LE -- use
iconv -f UTF-16LE -t UTF-8to read it DISPLAY=:99must be set (Xvfb virtual display number)
Deployment Pipeline
Use /mql5:mql5-ship slash command for full deployment:
--ea-onlyfor EA source changes--dll-onlyfor Rust DLL changes- Full pipeline: commit -> push -> pull on <gpu-host> -> compile -> validate
Detailed deployment steps (SSH commands, file copies, compilation) are in the headless-mt5-remote local skill. Not duplicated here to avoid drift.
Monitoring Commands
Check MT5 service status
systemctl --user status mt5
Check today's tick files
ls -la ~/.cache/opendeviationbar/ticks/FXVIEW_EURUSD/$(date +%Y)/
Count ticks in latest file
duckdb -c "SELECT count(*) FROM read_parquet('path/to/file.parquet')"
Check for recovery segments
ls ~/.cache/opendeviationbar/ticks/FXVIEW_EURUSD/$(date +%Y)/*_[0-9].parquet 2>/dev/null
View systemd service chain
systemctl --user list-units --type=service | grep -E "xvfb|xfce|x11vnc|mt5"
Check MT5 journal for errors
journalctl --user -u mt5 --since "1 hour ago" --no-pager
Scope Boundary
- This skill: Operations, monitoring, troubleshooting
headless-mt5-remotelocal skill: Deployment steps (SSH, file copy, compilation)fxview-parquet-consumerskill: Data consumption patterns (schema, DuckDB queries)
Troubleshooting
| Issue | Cause | Solution |
|---|---|---|
| MT5 service won't start | Display stack not running | Start xvfb first, then xfce, x11vnc, mt5 |
| EA not collecting ticks | Symbol not in Market Watch | Open chart for symbol in MT5, re-attach EA |
| Wine crash on PrintFormat | Using %I64u/%I64d specifiers | Replace with IntegerToString() |
| DLL load failure | Missing tick_writer.dll | Copy DLL to MQL5/Libraries/ (physical copy) |
| Gaps in tick data | EA restart or network issue | Check journalctl, verify recovery segments exist |
| Recovery file _1_2 | Normal after restart | All segments are valid, union in DuckDB |
| Compile log unreadable | UTF-16LE encoding | Use iconv -f UTF-16LE -t UTF-8 |
| Xvfb not running | Service crashed or not enabled | systemctl --user start xvfb |
Post-Execution Reflection
After this skill completes, check before closing:
- Did the command succeed? — If not, fix the instruction or error table that caused the failure.
- Did parameters or output change? — If the underlying tool's interface drifted, update Usage examples and Parameters table to match.
- Was a workaround needed? — If you had to improvise (different flags, extra steps), update this SKILL.md so the next invocation doesn't need the same workaround.
Only update if the issue is real and reproducible — not speculative.
How can the creator link this skill?
Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.
<a href="https://skillzs.dev/skills/terrylica/cc-skills/tick-collection-ops">View tick-collection-ops on skillZs</a>