Hey dudes,

Love Flask-Restless, but wanted to leave an issue here as I'd love to use the library, but a showstopper in my use case (and possibly that of other people) is that Flask-Restless really has no way (currently) to properly exclude database relationship queries with the pre and postprocessor stuff.

For instance, I have an application which has several models, namely:

State and Person. Each person has a state relationship (via ForeignKey).

My API clients need to frequently grab a list of all states in the system, so they'll do something like:

GET /api/states

And will immediately get their screen flooded with results (because Flask-Restless is traversing the backref and generating JSON for the millions of people in each state).

Even if I were to explicitly write a postprocessor for the GET_MANY hook, this wouldn't really solve the problem as each request to my API endpoint /api/states would still result in Flask-Restless spending an enormous amount of time loading my people backref and eventually discarding that data and returning a response.

I think a good solution to this problem is what issue #161 is suggesting: have Flask-Restless only load foreign key relationships when they're explicitly eagerly loaded by Flask-SQLAlchemy.

I'm documenting this here for discussion purposes. Thank you.

Hi Jeffrey

In #139 you are remove custom code for specifying include and exclude columns, after that it is very inconvenient to include/exclude columns in postprocessor function (particularly in GET_MANY postprocessor).

My suggestion is add new parameter "serialization_function" to create_api_blueprint and if this param is not None call this function instead of the _to_dict

I'm using flask_restless-1.0.0b2 and it seems that if I define a hybrid property like

    @hybrid_property
    def is_complete(self):
        """
        when all features are completed, this workorder should be completed
        """
        for f in self.features:
            if f.feature_status == 'Open':
                return 'No'
        return 'Yes'

    @is_complete.expression
    def is_complete(cls):
        return select([Workorder_Feature.feature_status]).\
                where(Workorder_Feature.workorder_id==cls.id)

And I pass parameters like this:

sort:is_complete

or

filter[objects]:[{"name":"is_complete","op":"like","val":"%No%"}]

It throws errors in flask_restless\search\operators.py on line 98 https://github.com/jfinkels/flask-restless/blob/master/flask_restless/search/operators.py#L98: 'Select' has no attribute 'like'

and in flask_restless\search\drivers.py on line 121 https://github.com/jfinkels/flask-restless/blob/master/flask_restless/search/drivers.py#L121: Select object has no attribute 'asc'

I noticed that by default Flask-Restless only returns one level of recursion on relationships. That is, a request to /api/model/ returns that models immediate attributes, the relationships and their immediate attributes, but does not return relationships on relationships or so forth.

It's possible to make a second request to pull these, such as /api/model//relationship which will then give you that second level, but each one will require setting up additional requests.

The main issue here is when we want to pull a big block of data and all the information we can on a model using asynchronous javascript, we have to code up several levels of asynchronous AJAX requests to get at the information. If we could specify how many levels of recursion we wanted with the first request, we could just get one big JSON blob in a single request containing all data. In my opinion, one much larger JSON blob may take longer to serve, but in a highly normalized database where there are many different joins and relationships going on, it'll be much preferred both for performance and coding (much fewer code to make a single AJAX call than multiple async calls, browsers tend to only send out 2-3 requests at once while waiting for responses, introducing latency as more and more ajax calls are stacked, etc).

I realize that probably the primary concern here is with infinite loops being setup if back references exist. The easiest solution would be to simply leave those out and not include those relationships during the recursion, since the information will be available higher up in the JSON hierarchy.

Also, depending on the amount of data and relationships for a given model, it could return more than is needed, so that is where a parameter specifying recursion level would be helpful.

The alternative to pragmatically determining when a back reference occurs would be an option to allow users to explicitly specify which relationships should be recursed somewhere. For example, users add something like recurse_relations attribute to the SQLAlchemy model that specifies which relationships can be recursed safely. It makes the work of figuring it out automatically a little easier, but does require extra work on the users since they must specify which ones should automatically recurse.

The junction mechanism doesn't solve the nested and/or problem.

Consider the query:

SELECT *
FROM table
WHERE fld1='val1' 
  AND (fld2='val2' OR fld3='val3' OR (val4='val4' AND val5='val5')) 
  AND fld6='val6';

The junction mechanism doesn't provide syntax to write it like a json filter. My method looks like an hack (because it's not intuitive) but works very well. It uses lists to group and apply the correct conjunction/disjunction. My implementation allow the WHERE part of the query to be written as:

{
    "filters": [
        {"name":"fld1", "op":"eq", "val":"val1"},
        [
            {"name":"fld2", "op":"eq", "val":"val2"},
            {"name":"fld3", "op":"eq", "val":"val3"},
            [
                {"name":"fld4", "op":"eq", "val":"val4"},
                {"name":"fld5", "op":"eq", "val":"val5"}
            ]
        ],
        {"name":"fld6", "op":"eq", "val":"val6"}
    ]
}

As you can see the or / and junction is toggled on every new level.

If you are interested in this approach I will write documentation and tests and submit and more clean pull request.

I needed a way to extend and modify default Flask-Restless functionality and came up with this. Basically idea is such:

• Before-action hooks allow more complex permission check and modifying/augmenting request
• After-action hooks allow to modify response
• Any extension hook can abort request or return a completely custom response

I'm not yet entirely sure about API of these hooks. For instance with _before_post or _after_get is clear enough - pass the model about to be created in case of POST and pass model + its data in dict in case of GET.

With _after_search, _before_patch, _after_patch is a bit more tricky. Now I think best approach is to pass along as much data as possible and let extending class make sense out of it.

This also makes post_form_preprocessor from #74 obsolete.

Here's some code from my project in progress (and more examples are in tests): https://gist.github.com/3155303

This change would also allow to address #2 rather elegantly. E.g.: https://gist.github.com/3155322

What do you think?

Hey, here's my solution for #476. I hope it's acceptable. If I've forgotten anything please let me know!

All enabled tests are passing on the branch:

[email protected]:~/work/flask-restless:overriden_primary_key_in_links  V $ py.test           (restless)
======================================= test session starts ========================================
platform darwin -- Python 2.7.10, pytest-2.9.0, py-1.4.31, pluggy-0.3.1
rootdir: /Users/awalton/work/flask-restless, inifile:
collected 449 items

tests/test_bulk.py sssssss
tests/test_creating.py ..........................................ss..
tests/test_deleting.py s............
tests/test_fetching.py ............sss................................................s...
tests/test_filtering.py s....................................................
tests/test_functions.py .........
tests/test_manager.py ......................s..
tests/test_metadata.py s.
tests/test_serialization.py ..................
tests/test_updating.py .................................sssss..
tests/test_updatingrelationship.py ..................................................
tests/test_validation.py ............
tests/test_jsonapi/test_creating_resources.py ........
tests/test_jsonapi/test_deleting_resources.py ..
tests/test_jsonapi/test_document_structure.py .....................
tests/test_jsonapi/test_fetching_data.py ...............................................
tests/test_jsonapi/test_server_responsibilities.py s......
tests/test_jsonapi/test_updating_relationships.py ..........
tests/test_jsonapi/test_updating_resources.py ............

============================= 426 passed, 23 skipped in 11.03 seconds ==============================

According to HTTP spec it is possible to create new resource by PUTting to its desired URI - http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.6 :

The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server. If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI.

And it's not possible with flask-restless as it requires table row to be already present:

# create a SQLAlchemy Query which has exactly the specified row
query = query_by_primary_key(self.session, self.model, instid)
if query.count() == 0:
    abort(404)

Enables explicit declaration of exposed model sub-resources by passing a relationname parameter to the APIManager.create_api method.

This enables model relation endpoints to declare their own include and exclude rules, methods, and post and pre processors.

If relationname is not passed to create_api, only the primary model endpoints are created (e.g. /api/person, /api/person/1) you won't be able to access other endpoints. Additional calls to create_api which include the relationname parameter create the relation endpoints and at this point include and exclude rules as well as post and pre processors can be passed (Same as the primary model or different).

This also enables filtering of exposed relations using the same API.

This introduces two backwards incompatible changes.

1. Each exposed relation endpoint needs to be defined using create_api and passing the relationname parameter.

2. When passing exclude_columns, if you wish to exclude all relational data in a field, you only need to pass the field name, and not each field under the relation. For example,

   exclude_columns = ['country']

   Instead of

   exclude_columns = ['country.name', 'country.code']

This commit also updates the has_field doc string, which should also work for hybrid_property fields.

An example declaration exposing a primary resource and a sub-resource:

apimanager.create_api(Country,
        methods=methods,
        url_prefix="/api/v1",
        results_per_page=results_per_page,
        max_results_per_page=100000,
        postprocessors=postprocessors_,
        preprocessors=preprocessors_,
        allow_functions=allow_functions,
        exclude_columns=[],
        include_columns=[],
        validation_exceptions=[ValidationError,
            ProcessingException,
            BadRequest],
        )

apimanager.create_api(Country,
        relationname='cities',
        methods=methods,
        url_prefix="/api/v1",
        results_per_page=results_per_page,
        max_results_per_page=100000,
        postprocessors=postprocessors_,
        preprocessors=preprocessors_,
        allow_functions=allow_functions,
        exclude_columns=[],
        include_columns=[],
        validation_exceptions=[ValidationError,
            ProcessingException,
            BadRequest],
        )

I'm getting

TypeError: <lambda>() got an unexpected keyword argument 'page'

upon running the example quickstart.py with Python 3.4. It's also broken on Python 2.7. I consider this a problem of this library. It's only a problem since the commit that merges in the mimerender pull request.

I am overlooking something simple. I am making an GET request to a model called "event", trying to get only back only subset of the events in the database. However, it returns the whole list. Below is the header information.

Request URL:http://localhost:5000/api/event?limit=2&offset=2 Request Method:GET Status Code:200 OK Request Headersview source Accept:application/json, text/plain, / Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3 Accept-Encoding:gzip,deflate,sdch Accept-Language:en-US,en;q=0.8 Cache-Control:no-cache Connection:keep-alive Cookie:csrftoken=O1TcJZ3teDRPyT0DqEqjOeve12mJes55 Host:localhost:5000 Pragma:no-cache Referer:http://localhost:5000/ User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.52 Safari/537.17 X-Requested-With:XMLHttpRequest Query String Parametersview sourceview URL encoded limit:2 offset:2 Response Headersview source Content-Length:6893 Content-Type:application/json Date:Tue, 22 Jan 2013 23:33:54 GMT Server:Werkzeug/0.8.3 Python/2.7.2

In my flask restless file, I have a read user and a write user. Each has their own db session. This is basically what I'm trying to do at the moment:

manager = flask_restless.APIManager(app, session=read_session)

server = manager.create_api(Server, methods=["GET"])

admin_manager = flask_restless.APIManager(app, session=write_session)

server_admin = admin_manager.create_api(Server, collection_name='server_admin', methods=["GET","POST","PUT","PATCH","DELETE"])

but am getting an error that says: "Flask-Restless has already been initialized on this application"

I also tried this and it doesn't seem to work either

manager = flask_restless.APIManager(app, session=read_session)

server = manager.create_api(Server, methods=["GET"])

manager.session = write_session

server_admin = manager.create_api(Server, collection_name='server_admin', methods=["GET","POST","PUT","PATCH","DELETE"])

Flask-restless looks excellent, but I really like the browsable API offered by flaskAPI. The ability to send PUT/POST requests directly from the browsable API instead of postman/curl/etc. is appealing.

The issue is that the views that flask-restless returns out of the box are incompatible with what FlaskAPI expects, so FlaskAPI will ignore routes handled by flask-restless. See this comment from the FlaskAPI developer - https://github.com/flask-api/flask-api/issues/92

What I'm hoping to find is some option (or small hack) for flask-restless that will make it return a list or dictionary instead of json/html, and FlaskAPI would then handle returning html or json depending on the content type requested. Is that possible with flask-restless? So far I haven't had any luck finding anything in the docs that looks relevant.

Thanks in advance.

In a table A that has a list of objects B, is it possible to append an object? The alternative would be to retrieve the list, add an object to it, and upload the modified list. The problem with this solution is that the local object A could become stale between the GET and the PATCH. Making the whole system inconsistent.

Using an include results in fetching every object individually, even when this is not necessary. This commit evaluates the given query and uses the resulting objects instead. If your data model is properly configured, the important objects will already be fetched in this query. This improves some of our endpoint time from > 1 minute to 1 second.