Skip to content

workhorse: Improve perf for LSIF hover entries with markdown

Arran Walker requested to merge ajwalker/code-hover-lang-perf into master

What does this MR do and why?

We recently upgraded our version of lsif-go for the Runner project and broke the upload processing.

The main reason is that lsif-go has switched over to Markdown code hover entries and we're now exceeding the 100MB artifact upload limit.

However, whilst debugging this problem, I found that Workhorse currently takes ~72 seconds to parse the LSIF file for the Runner project. This is due to the lexer library wasting time looking for a lexer that doesn't exist when no language is specified (which is always the case for markdown-like entries).

This change avoids the slow path when a language hasn't been specified or is unsupported by the lexer. With this, the parse time for the Runner project's LSIF was lowered to ~4 seconds.

Benchmarks

Before

goos: darwin
goarch: arm64
pkg: gitlab.com/gitlab-org/gitlab/workhorse/internal/lsif_transformer/parser
BenchmarkHighlight/lang:go-10         	   57937	     20579 ns/op	    7072 B/op	     131 allocs/op
BenchmarkHighlight/lang:ruby-10       	   24476	     49108 ns/op	    8112 B/op	     154 allocs/op
BenchmarkHighlight/lang:-10           	     538	   2219470 ns/op	     936 B/op	      17 allocs/op
BenchmarkHighlight/lang:zzz-10        	     411	   2959513 ns/op	     848 B/op	      17 allocs/op

After

goos: darwin
goarch: arm64
pkg: gitlab.com/gitlab-org/gitlab/workhorse/internal/lsif_transformer/parser
BenchmarkHighlight/lang:go-10         	   57897	     20567 ns/op	    7072 B/op	     131 allocs/op
BenchmarkHighlight/lang:ruby-10       	   24544	     48779 ns/op	    8112 B/op	     154 allocs/op
BenchmarkHighlight/lang:-10           	  382830	      3099 ns/op	     936 B/op	      17 allocs/op
BenchmarkHighlight/lang:zzz-10        	  626169	      1924 ns/op	     848 B/op	      17 allocs/op

Describe in detail what your merge request does and why.

Relates to LSIF: Support markdown-like content format (!63213 - merged)

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Arran Walker

Merge request reports