Logs from files

Many applications write their logs to files. The OpenTelemetry Collector’s filelog receiver reads those files as they grow, turns each line into a log record and sends it to FuseScope. It ships in the otelcol-contrib distribution.

Use this page when an application or a service on a VM or in Docker writes logs to files, and you can’t or don’t want to change it to send logs over OTLP.

On Kubernetes, use the Kubernetes logs preset instead: it already reads every pod’s log files.

How it works

1. A collector runs next to the application, as a container or as a service on the host. See Collector in Docker or Collector on a VM to install it.
2. The filelog receiver watches the files that match include and reads new lines.
3. Optional operators parse each line into fields, a timestamp and a severity.
4. An exporter sends the logs to the FuseScope collector over OTLP.

A minimal configuration

This reads every .log file under /var/log/checkout, names the service checkout and sends the logs to FuseScope over OTLP/HTTP:

config.yaml

receivers:
  filelog:
    include:
      - /var/log/checkout/*.log
    exclude:
      - /var/log/checkout/*.gz
    start_at: end
    include_file_path: true
    resource:
      service.name: checkout
      deployment.environment.name: production

processors:
  batch: {}

exporters:
  otlphttp:
    endpoint: http://fusescope-collector.example.com:4318

service:
  pipelines:
    logs:
      receivers: [filelog]
      processors: [batch]
      exporters: [otlphttp]

Replace fusescope-collector.example.com with your FuseScope collector address. See OTLP endpoints for the address and TLS settings.

Receiver options

Option	Default	What it does
include	required	Glob patterns of the files to read, such as /var/log/app/*.log.
exclude	[]	Glob patterns to skip, checked against the files include matched.
start_at	end	Where to start in a file the receiver has not seen before: end (new lines only) or beginning (the whole file).
include_file_name	true	Adds the file name as the attribute log.file.name.
include_file_path	false	Adds the full path as the attribute log.file.path.
resource	{}	Resource attributes added to every log, such as service.name.
attributes	{}	Log attributes added to every log.
storage	none	A storage extension that keeps read positions across restarts. See Keep your place across restarts.
multiline	none	Joins several lines into one log. See Multiline logs.
operators	[]	Steps that parse each log. See Parse fields.

The full list is in the filelog receiver README.

start_at: end skips old lines

With the default start_at: end, the receiver reads only lines written after it starts. To load what a file already holds, set start_at: beginning once, together with a storage extension so a restart doesn’t read it again.

Keep your place across restarts

Without storage, the receiver keeps its read positions in memory. After a restart it forgets them: with start_at: end it misses lines written while it was down, and with start_at: beginning it reads every file again and sends duplicates.

The file_storage extension saves the positions to disk. Point the receiver’s storage at it:

config.yaml

extensions:
  file_storage:
    directory: /var/lib/otelcol/file_storage
    create_directory: true

receivers:
  filelog:
    include:
      - /var/log/checkout/*.log
    start_at: beginning
    storage: file_storage

service:
  extensions: [file_storage]
  pipelines:
    logs:
      receivers: [filelog]
      processors: [batch]
      exporters: [otlphttp]

The collector must be able to write to directory, and the directory must survive a restart: on Docker, mount it as a volume.

Multiline logs

By default each line is one log. A Java stack trace then arrives as dozens of separate logs. multiline tells the receiver where a new log starts, so the lines that follow are joined to it:

/var/log/checkout/app.log

2026-10-23 14:02:11.532 ERROR [http-nio-8080-exec-3] c.e.checkout.PaymentService - Payment failed
java.lang.IllegalStateException: card declined
  at com.example.checkout.PaymentService.charge(PaymentService.java:88)
  at com.example.checkout.OrderController.place(OrderController.java:41)
2026-10-23 14:02:12.004 INFO [http-nio-8080-exec-4] c.e.checkout.OrderController - Order 1042 placed

Every log starts with a date, so:

receivers:
  filelog:
    include:
      - /var/log/checkout/*.log
    multiline:
      line_start_pattern: ^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}

This produces two logs: the error with its whole stack trace, and the info line. multiline takes exactly one of line_start_pattern or line_end_pattern.

Parse fields

Operators run in order on each log, inside the receiver. Each has a type; give an id when you use the same type twice. Field paths start with body, attributes or resource; use brackets for keys with dots, such as resource["service.name"].

JSON lines

For a file with one JSON object per line:

{"ts":"2026-10-23T14:02:11.532Z","level":"error","service":"checkout","msg":"Payment failed","order_id":1042}

receivers:
  filelog:
    include:
      - /var/log/checkout/*.json
    operators:
      - type: json_parser
        timestamp:
          parse_from: attributes.ts
          layout_type: gotime
          layout: "2006-01-02T15:04:05.000Z07:00"
        severity:
          parse_from: attributes.level
      - type: move
        from: attributes.service
        to: resource["service.name"]
      - type: move
        from: attributes.msg
        to: body

json_parser turns the keys into attributes. The first move makes the service field the log’s service.name; the second makes msg the log body.

Plain text with a regular expression

For the Java log above, regex_parser splits each log into named groups. (?s) lets the last group span the joined stack-trace lines:

receivers:
  filelog:
    include:
      - /var/log/checkout/*.log
    multiline:
      line_start_pattern: ^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}
    resource:
      service.name: checkout
    operators:
      - type: regex_parser
        regex: '(?s)^(?P<time>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\.\d{3}) (?P<level>[A-Z]+) +\[(?P<thread>[^\]]+)\] (?P<logger>\S+) - (?P<message>.*)$'
        timestamp:
          parse_from: attributes.time
          layout: "%Y-%m-%d %H:%M:%S.%L"
          location: UTC
        severity:
          parse_from: attributes.level
      - type: move
        from: attributes.message
        to: body

Block	Field	What it does
timestamp	parse_from, layout	Sets the log time from a field. layout_type is strptime (default), gotime or epoch.
timestamp	location	Time zone for timestamps that carry none, such as UTC or America/Sao_Paulo. Defaults to the host’s local zone.
severity	parse_from	Sets the severity from a field. Common values such as ERROR, warn and info map without a mapping.

By default a parser that fails to match keeps the log unchanged (on_error: send).

See the stanza operator docs for every operator.

Set service.name

FuseScope groups logs by the service.name resource attribute. Without it, logs show under an unknown service and you can’t filter them per service. Set it in one of two ways:

• One service per receiver: add it with the receiver’s resource block, as in the examples above. Use a separate receiver, such as filelog/checkout and filelog/billing, for each service.
• From the log itself: parse a field and move it to resource["service.name"], as in JSON lines.

Parse in the collector or in FuseScope

You can also parse in FuseScope with log pipelines, built and previewed in the UI.

Parse in the collector when	Parse in a FuseScope log pipeline when
You need multiline: lines must be joined where they are read.	You want to try and preview changes in the UI without redeploying agents.
You set service.name or other resource attributes per file.	Several sources send the same format and should be parsed the same way.
You want to drop or reduce logs before they leave the host.	The people who parse logs don’t manage the agents.

A common split: multiline and service.name in the collector, field extraction and severity in a pipeline.

Run it

Docker

Mount the log directory read-only and a host directory for the read positions. The image runs as user 10001, so it must be able to read the logs and write the storage directory.

sudo mkdir -p /var/lib/otelcol-storage
sudo chown 10001:10001 /var/lib/otelcol-storage

docker run -d --name otelcol \
  -v "$PWD/config.yaml:/etc/otelcol-contrib/config.yaml:ro" \
  -v /var/log/checkout:/var/log/checkout:ro \
  -v /var/lib/otelcol-storage:/var/lib/otelcol/file_storage \
  otel/opentelemetry-collector-contrib:0.139.0

To read another container’s logs, write them to a shared volume and mount it the same way. See Collector in Docker.

VM

Install the otelcol-contrib package, put the configuration in /etc/otelcol-contrib/config.yaml and restart the service. The service runs as the otelcol-contrib user, which must be able to read the log files:

sudo usermod -aG adm otelcol-contrib
sudo mkdir -p /var/lib/otelcol/file_storage
sudo chown otelcol-contrib:otelcol-contrib /var/lib/otelcol/file_storage
sudo systemctl restart otelcol-contrib

The adm group reads most of /var/log on Debian and Ubuntu; for other files, grant read access to the group that owns them. See Collector on a VM.

Check that data arrives

1. Write a line to a watched file, such as echo "$(date '+%F %T.000') ERROR [main] test - hello FuseScope" >> /var/log/checkout/app.log.
2. Open the Logs explorer and filter by service.name = checkout.
3. If nothing arrives, check the collector’s own logs for errors from filelog (often a permission error) or from the exporter.

Related

• Collector in Docker and Collector on a VM: install the agent.
• Log pipelines: parse logs in FuseScope.
• Kubernetes logs: pod logs with the k8s-infra chart.
• Logs explorer: search the logs.