Addressed issue #26 in this PR. Along with that, I've made some additional changes too.

1. Added pytest dependency into requirements.txt.
2. Minor correction in README.md.

This is my very first contribution to an external open source project. Feel free to comment on the contribution and I'll be glad to learn and improve. Thank you.

I can generate correct Prometheus metrics with gunicorn when it's worker is only 1. (I check the gauge is correct) But when I increase the worker to 2, the gauge is not correct anymore.

And I add this environment, the result is the metric page is empty now. export PROMETHEUS_MULTIPROC_DIR=/home/ubuntu/ap/tmp

I also try to add the code, but it's still show empty. Any suggestion?

from prometheus_client import multiprocess

def child_exit(server, worker):
    multiprocess.mark_process_dead(worker.pid)

When using code like the example below (with fastapi in this case)

from starlette_exporter import PrometheusMiddleware, handle_metrics
from http import HTTPStatus
from fastapi import FastAPI, HTTPException, Response
import uvicorn

app = FastAPI()
app.add_middleware(PrometheusMiddleware)
app.add_route("/metrics", handle_metrics)


@app.get("/")
async def root():
    return Response(status_code=HTTPStatus.OK)


@app.get("/200")
async def root():
    return {"I am returning 200"}


@app.get("/500")
async def root():
    raise HTTPException(status_code=500)
    return {"I am returning 200"}


@app.get("/500v2")
async def root():
    raise HTTPException(status_code=HTTPStatus.INTERNAL_SERVER_ERROR)
    return {"I am returning 200"}


if __name__ == "__main__":
    uvicorn.run(app, host="127.0.0.1", port=5000, log_level="info")

The status code that is displayed in the metrics is the enum name, not the numeric code.

starlette_request_duration_seconds_bucket{app_name="starlette",le="10.0",method="GET",path="/",status_code="HTTPStatus.OK"} 2.0

Is this desired behaviour?

Perhaps it might be better to attempt to convert the status code to an int?

I think all we would have to do is maybe do a type check and convert to int right here https://github.com/stephenhillier/starlette_exporter/blob/master/starlette_exporter/middleware.py#L174

HI again ! Last issue for the period I believe ;-)

Would you mind adding a quick doc about how to extend the metrics ? The FastAPI / starlette ones are a very good basis, but I'ld like to add some related to my app.

For example I have a Postgres database, I want to add the number of active subscription in the metrics, aka the result of a SELECT * FROM subscription WHERE active == 1 that would show up as starlette_subscription_active{app_name="starlette",query="SELECT * FROM subscription WHERE active == 1"} 1382

Base by request on https://github.com/stephenhillier/starlette_exporter/issues/45 added an bool feature to enable to add hostname/url to the path value. Enable:

app.add_middleware(starelette_exporter.Prometheus, hn_ext=True)

Output: starlette_requests_total{app_name="starlette",method="GET",path="<HTTP hearder host>/200",status_code="200"} 1.0

This header is always on as pre http1.1 headers. https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html [14.23] Host

A small set of improvements — I can split them up into separate PRs if you need it.

Each commit is self-contained

1. Change middleware to be a generic ASGI3 middleware. Starlette is discouraging use of BaseHTTPMiddleware because it causes all sorts of problems with streaming requests & responses. eg: https://github.com/encode/starlette/issues/919#issuecomment-672908610 and others. Personally, I'm running into early EOFs during reading request.stream() when PrometheusMiddleware (or anything BaseHTTPMiddleware-derived) is active.

2. Use time.perf_counter() instead of time.time(). It doesn't go backwards and it operates at a higher precision

3. Add prefix option for metric naming #3

Waiting proposal for naming .. on optional_metrics Now we have 2 optional metrics request_response_body_size => how much bytes the server send back to the client. client_receive_body_size => how much data receive server from client (post/put)

It will be great to have a better naming .

Added optional_metrics = ["all"] will work on rec and snd body_size Added receive_body_size metric Added starting Prometheus as service on different port .

I think something wrong when request method is OPTIONS like below

starlette_requests_total{app_name="starlette",method="GET",path="/api/v1/datasets",status_code="200"} 92.0
starlette_requests_total{app_name="starlette",method="GET",path="/api/v1/tasks/nm",status_code="200"} 53.0
starlette_requests_total{app_name="starlette",method="GET",path="/api/v1/datasets/{did}",status_code="200"} 22.0
starlette_requests_total{app_name="starlette",method="GET",path="/api/v1/datasets/{did}/stats",status_code="200"} 7.0
starlette_requests_total{app_name="starlette",method="DELETE",path="/api/v1/datasets/{did}",status_code="200"} 4.0
starlette_requests_total{app_name="starlette",method="OPTIONS",path="/api/v1/datasets/567738610",status_code="200"} 1.0
starlette_requests_total{app_name="starlette",method="OPTIONS",path="/api/v1/datasets/2003501303",status_code="200"} 1.0
starlette_requests_total{app_name="starlette",method="OPTIONS",path="/api/v1/datasets/921436406",status_code="200"} 1.0
starlette_requests_total{app_name="starlette",method="OPTIONS",path="/api/v1/datasets/799719666",status_code="200"} 1.0
starlette_requests_total{app_name="starlette",method="OPTIONS",path="/api/v1/datasets/1602879743",status_code="200"} 1.0
starlette_requests_total{app_name="starlette",method="OPTIONS",path="/api/v1/datasets/150110457",status_code="200"} 1.0
starlette_requests_total{app_name="starlette",method="OPTIONS",path="/api/v1/datasets/1256292570",status_code="200"} 1.0

My version is starlette-exporter==0.11.0 and I add prometheus middleware

app.add_middleware(PrometheusMiddleware, group_paths=True)
app.add_route("/metrics", handle_metrics)

Is there any problem? Please take a look

Adds an option to avoid collecting metrics on specific paths. This option is useful to avoid the collection on endpoints used for healthcheck, liveness or readiness probes and even to avoid collecting metrics in the self Prometheus exposed endpoint (commonly /metrics).

The new option is an optional list called skip_paths.

How to define custom labels that will be used on every starlette_exporter metrics ? For example to add the user_id (email) extracted from a JWT token or any part of the request/session.

It could be acheive with a callback or by monkey-patching *.

Does it make sense ?

https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#exemplars-1

https://github.com/prometheus/client_python/blob/f17a8361ad3ed5bc47f193ac03b00911120a8d81/README.md#exemplars

Would it be reasonable to have optional dependencies on open telemetry to get the trace info for a bucket?

I can use starlette_exporter to generate starlette_request_*** metrics now, but I don't see python_*** metrics. Is there a flag to enable it? (Sorry, I cannot find related information in issues and code... If I miss anything, please let me know, thanks.)

I am running FastAPI (0.79.1), starlette (0.19.1), satrlette_exporter (0.14.0), uvicorn (0.17.6) and prometheus_client (0.14.1). When I set root_path to fix my /docs (It is behind nginx), most of my application metrics disappeared from /metrics. I see this was supposed to be fixed in v0.12.0, so perhaps I am missing something? Thanks for any help or insight on this.

I am still getting a bit of a handle on how python async works however I have a question.

Since starlette_exporter depends on the client_python which is not generally async am I blocking the event loop when I enable client_python's multi-process mode?

I can see it uses the file system here to store/share its metrics between workers and in doing so makes regular file open calls.

Do these open calls block the event loop?

Often in a production application one might want to hide the /metrics endpoint from public traffic. Using the methods listed in the README, one would have to explicitly mask the /metrics route within a reverse proxy and bind to two different ports which can be non-trivial (see encode/uvicorn#571 for context).

In my experience I've found it easier to just expose the metrics on a separate port (Ie. 9090) via Prometheus's default start_http_server function but I'm not sure if this is supported by starlette-exporter. This way the metrics requests are served completely internally (and, for example, can only be exposed internally within a kubernetes cluster). While probably not necessary, to be clean I also integrated the server used in start_http_server with starlette's lifespan events (otherwise I'm worried for example the socket won't unbind for some period of time when hot reloading).

My questions are (edit: updated questions):

1. Is this supported/possible?
2. Would an example that uses start_http_server be accepted into the README?
3. Would code that calls start_http_server handling lifespan hooks be accepted as a contribution?

cc @NargiT

If the match is partial we want the path of the route and not None, which would result in a dropped metric if filter_unhandled_paths is set to True and a request is made to an actually handled endpoint but with the wrong http method.