Handle panic in the store goroutine. (#2774)
Handle panic in the store goroutine. (#2774)

* Adds logfmt, regexp and json logql parser

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* hook the ast with parsers.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* hook parser with memchunk.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* hook parser with the storage.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* hook parser with ingesters

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* fixes all tests

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Refactor to pipeline and implement ast parsing.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Fixes the lexer for duration and range

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Fixes all tests and add some for label filters

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Add label and line format.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Add tests for fmt label and line with validations.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Polishing parsers and add some more test cases

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Finish the unwrap parser, still need to add more tests

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Indent this hell.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Moar tests and it works.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Add more tests which lead me to find a bug in the lexer

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Add more tests and fix all engine tests

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Fixes match stage in promtail pipelines.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Hook Pipeline into ingester, tailer and storage.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Correctly setup sharding for logqlv2

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Fixes precedences issue with label filters and add moar tests :v:

* Adds quantile_over_time, grouping for non associate range aggregation parsing and moar tests

* Extract with grouping

* Adds parsing duration on unwrap

* Improve the lexer to support more common identifier as functions.

Also add duration convertion for unwrap.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Fixes the frontend logs to include org_id.

The auth middleware was happening after the stats one and so org_id was not set :facepalm:.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Support byte sizes in label filters.

This patch extends the duration label filter with support for byte sizes
such as `1kB` and `42MiB`.

* Wip on error handling.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Fixes json parser with prometheus label name rules.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* fixup! Support byte sizes in label filters.

* Wip error handling, commit before big refactoring.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Refactoring in progress.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Work in progress.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Got something that builds and throw __error__ labels properly now.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Add error handling + fixes groupins and post filtering.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* 400 on pipeline errors.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Fixes a races in the log pipeline.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Unsure the key is parsable and valid.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Cleanup and code documentation.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Lint.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Lint.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Fixes frontend handler.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Fixes old test.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Fix go1.15 local failing test.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Fixes race conditions in the batch iterator.

We should never advance an iterator in parallel. Unfortunately before the code was building iterators while advancing previous one, building iterator can advance iterator and thus creates a race condition. This changeset make sure we only fetch chunks in advance and build iterator and iterate over them in sequence.

Also add support for labels in the cacheIterator which is required for logqlv2.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

* Handle panic in the store goroutine.

This could cause Loki to crash if a panic happens in the store since it was happening in another goroutine.

Signed-off-by: Cyril Tovena <1f3b43b4f8eb8a957255fd6382cb431f49248e69@gmail.com>

Co-authored-by: Karsten Jeschkies <13fbd79c3d390e5d6585a21e11ff5ec1970cff0c@jeschkies.xyz>
grafana/loki,grafana/loki,grafana/loki,grafana/loki,grafana/loki,grafana/loki