1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
|
journaldriver
=============
This is a small daemon used to forward logs from `journald` (systemd's
logging service) to [Stackdriver Logging][].
Many existing log services are written in inefficient dynamic
languages with error-prone "cover every possible use-case"
configuration. `journaldriver` instead aims to fit a specific use-case
very well, instead of covering every possible logging setup.
`journaldriver` can be run on GCP-instances with no additional
configuration as authentication tokens are retrieved from the
[metadata server][].
<!-- markdown-toc start - Don't edit this section. Run M-x markdown-toc-refresh-toc -->
**Table of Contents**
- [Features](#features)
- [Usage on Google Cloud Platform](#usage-on-google-cloud-platform)
- [Usage outside of Google Cloud Platform](#usage-outside-of-google-cloud-platform)
- [Log levels / severities / priorities](#log-levels--severities--priorities)
- [NixOS module](#nixos-module)
- [Stackdriver Error Reporting](#stackdriver-error-reporting)
<!-- markdown-toc end -->
# Features
* `journaldriver` persists the last forwarded position in the journal
and will resume forwarding at the same position after a restart
* `journaldriver` will recognise log entries in JSON format and
forward them appropriately to make structured log entries available
in Stackdriver
* `journaldriver` can be used outside of GCP by configuring static
credentials
* `journaldriver` will recognise journald's log priority levels and
convert them into equivalent Stackdriver log severity levels
# Usage on Google Cloud Platform
`journaldriver` does not require any configuration when running on GCP
instances.
1. Install `journaldriver` on the instance from which you wish to
forward logs.
2. Ensure that the instance has the appropriate permissions to write
to Stackdriver. Google continously changes how IAM is implemented
on GCP, so you will have to refer to [Google's documentation][].
By default instances have the required permissions if Stackdriver
Logging support is enabled in the project.
3. Start `journaldriver`, for example via `systemd`.
# Usage outside of Google Cloud Platform
When running outside of GCP, the following extra steps need to be
performed:
1. Create a Google Cloud Platform service account with the "Log
Writer" role and download its private key in JSON-format.
2. When starting `journaldriver`, configure the following environment
variables:
* `GOOGLE_CLOUD_PROJECT`: Name of the GCP project to which logs
should be written.
* `GOOGLE_APPLICATION_CREDENTIALS`: Filesystem path to the
JSON-file containing the service account's private key.
* `LOG_STREAM`: Name of the target log stream in Stackdriver Logging.
This will be automatically created if it does not yet exist.
* `LOG_NAME`: Name of the target log to write to. This defaults to
`journaldriver` if unset, but it is recommended to - for
example - set it to the machine hostname.
# Log levels / severities / priorities
`journaldriver` recognises [journald's priorities][] and converts them
into [equivalent severities][] in Stackdriver. Both sets of values
correspond to standard `syslog` priorities.
The easiest way to emit log messages with priorites from an
application is to use [priority prefixes][], which are compatible with
structured log messages.
For example, to emit a simple warning message (structured and
unstructured):
```
$ echo '<4>{"fnord":true, "msg":"structured log (warning)"}' | systemd-cat
$ echo '<4>unstructured log (warning)' | systemd-cat
```
# NixOS module
The NixOS package repository [contains a module][] for setting up
`journaldriver` on NixOS machines. NixOS by default uses `systemd` for
service management and `journald` for logging, which means that log
output from most services will be captured automatically.
On a GCP instance the only required option is this:
```nix
services.journaldriver.enable = true;
```
When running outside of GCP, the configuration looks as follows:
```nix
services.journaldriver = {
enable = true;
logStream = "prod-environment";
logName = "hostname";
googleCloudProject = "gcp-project-name";
applicationCredentials = keyFile;
};
```
**Note**: The `journaldriver`-module is included in stable releases of
NixOS since NixOS 18.09.
# Stackdriver Error Reporting
The [Stackdriver Error Reporting][] service of Google's monitoring
toolbox supports automatically detecting and correlating errors from
log entries.
To use this functionality log messages must be logged in the expected
[log format][].
*Note*: Reporting errors from non-GCP instances requires that the
`LOG_STREAM` environment variable is set to the special value
`global`.
This value changes the monitored resource descriptor from a log stream
to the project-global stream. Due to a limitation in Stackdriver Error
Reporting, this is the only way to correctly ingest errors from
non-GCP machines. Please see [issue #4][] for more information about
this.
[Stackdriver Logging]: https://cloud.google.com/logging/
[metadata server]: https://cloud.google.com/compute/docs/storing-retrieving-metadata
[Google's documentation]: https://cloud.google.com/logging/docs/access-control
[NixOS]: https://nixos.org/
[contains a module]: https://github.com/NixOS/nixpkgs/pull/42134
[journald's priorities]: http://0pointer.de/public/systemd-man/sd-daemon.html
[equivalent severities]: https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#logseverity
[priority prefixes]: http://0pointer.de/public/systemd-man/sd-daemon.html
[Stackdriver Error Reporting]: https://cloud.google.com/error-reporting/
[log format]: https://cloud.google.com/error-reporting/docs/formatting-error-messages
[issue #4]: https://github.com/tazjin/journaldriver/issues/4
|
