I'd like to take a look at implementing this. I'm reading up on the Jaeger project and open traces and then should have a plan.

Great @nwneisen :) Take your time for research, then we can sync here and agree about implementation.

@nwneisen fantastic!

What would be great also is that - once this is done - you document it through a blogpost (with nice screenshots), so that we have a documentation for other developers how to use OpenTarcing with Mainflux.

Your text will be then featured on Mainflux official blog: https://medium.com/mainflux-iot-platform (you can ping @Sale72 to help you with the draft process).

Also, it would be great to add a chapter in the official documentation as well.

After playing around with Jaeger and its examples, I think I've got my head wrapped around traces.

I found were spans are currently being created in mainflux. One such location is things/tracing/things.go. Here span.SetTag() can be used to add client information such as component name, span kind, and any errors. The transaction name (Save, Update, etc) also seems like useful information at this point but I don't see anything for it in the conventions document above.

From here the span will be passed to the database code at things/postgres/things.go via the context parameter. The span can be pulled out and have tags such as the query string and parameters added here. There are also some error cases that could be added to the span logs.

@nwneisen yes, that is basically what we would like to have. The query string will be problematic to add to the span because driver creates end SQL statement that will be sent to the PostgreSQL under the hood. There are some interceptors (such as this one) that are being developed, but they are not stable IMHO and shouldn't be used in Mainflux. I'm not sure how we should approach this SQL statement logging problem.

I pushed a draft of my current changes. While I made sure everything built, make rundev seems to be a bit different than make run. I'll be taking a look at the differences so I can pull up Jaeger and trigger spans.

I put the code in place to pull the span from the context. Right now it it adds the SQL query before it has values in place. I can only think of two options for the sql.statement:

1. Add the query string without values (what is in place now) along with tags for each individual value.
2. Build the query string manually.

The first option would not follow the open tracing recommendations but would provide the same tracing information until an interceptor looks more stable. The second option doesn't really seem like an option as I think it suggests it is the final query to be ran on the database when it is not.

Finished this issue in #875 and #869. I just need to talk with @Sale72 about the draft process.

@nwneisen, @anovakovic01, @drasko as #869 has been merged, can this issue be closed or is there something more to do?

@chombium we weren't able to add all of the tags suggested in the specification. This might be still open to keep those additional tags in mind to be added later.