For those who haven’t used Meltano before, this is how I got it work with self hosted Gitlab (hosted by your organisation or yourself).
To install Meltano – follow the instructions that can be found at https://meltano.com. The better method to install Meltano is via virtualenv on Python. If you are stuck with Windows, perhaps Docker is an option. Note that Meltano has pivoted focus to improve their open source offering so things are not that great in the UI world as of version 13.6.1. The command line is the recommended tool.
meltano add transformer dbt
meltano add extractor --include-related tap-gitlab
meltano add loader target-postgres
meltano config tap-gitlab set api_url "<<your customer server base url>>"
After doing that, you need to edit the .env file in your project directory and set the following parameters. Note that the values depend on your environment, these are merely placeholders.
export PG_ADDRESS="localhost"
export PG_USERNAME="postgres"
export PG_PORT="5432"
export PG_PASSWORD="postgres"
export PG_DATABASE="meltano"
export GITLAB_API_TOKEN="<<your Gitlab API access token>>"
export GITLAB_API_GROUPS="<<space separated list of groups>>"
export GITLAB_API_PROJECTS=""
export GITLAB_API_START_DATE="2019-01-01T00:00:00Z"
Next, edit the meltano.yml file and add the following to the vars section of transforms for tap-gitlab.
entry_table: "{{ env_var('PG_SCHEMA') }}.entry"
generationmix_table: "{{ env_var('PG_SCHEMA') }}.generationmix"
region_table: "{{ env_var('PG_SCHEMA') }}.region"
Once you are done with these settings, execute
meltano elt tap-gitlab target-postgres --transform run
After that runs successfully, you can run
meltano ui
and browse the reports at http://localhost:5000