sign_in IntegrationHelper produces errors when trying to load a session #4696

RocketPuppy · 2017-11-07T16:25:33Z

Rails 5 API mode, devise cookie auth works just fine in the actual application, meaning all middleware and initializers are set up properly.

I have an ApiController that looks like this:

class ApiController < ActionController::API
end

And all my controllers inherit from it. I have multiple user scopes. The integration test helpers are set up according to the documentation:

# test/test_helper.rb

class ActionDispatch::IntegrationTest
  include Devise::Test::IntegrationHelpers
end

In a test:

class FooControllerTest < ActionDispatch::IntegrationTest
  test "should be successful" do
    sign_in CustomUser.first
    get foo_url, xhr: true, as: :json

    assert(@response.successful?) # fail
  end
end

In the response body I get this error message, implying things are failing even before they get to my controllers.

> undefined method `[]=' for nil:NilClass

actionpack (5.1.4) lib/action_dispatch/request/session.rb, line 217
-------------------------------------------------------------------


  212             load! unless loaded?
  213           end
  214
  215           def load!
  216             id, session = @by.load_session @req
> 217             options[:id] = id
  218             @delegate.replace(stringify_keys(session))
  219             @loaded = true
  220           end
  221
  222           def stringify_keys(other)


App backtrace
-------------

 - actionpack (5.1.4) lib/action_dispatch/request/session.rb:217:in `load!'
 - actionpack (5.1.4) lib/action_dispatch/request/session.rb:212:in `load_for_write!'
 - actionpack (5.1.4) lib/action_dispatch/request/session.rb:197:in `merge!'
 - actionpack (5.1.4) lib/action_dispatch/request/session.rb:17:in `create'
 - actionpack (5.1.4) lib/action_dispatch/middleware/session/abstract_store.rb:71:in `prepare_session'
 - rack (2.0.3) lib/rack/session/abstract/id.rb:231:in `context'
 - rack (2.0.3) lib/rack/session/abstract/id.rb:226:in `call'
 - actionpack (5.1.4) lib/action_dispatch/middleware/cookies.rb:613:in `call'
 - warden (1.2.7) lib/warden/manager.rb:36:in `block in call'
 - warden (1.2.7) lib/warden/manager.rb:35:in `catch'
 - warden (1.2.7) lib/warden/manager.rb:35:in `call'
 - rack (2.0.3) lib/rack/etag.rb:25:in `call'
 - rack (2.0.3) lib/rack/conditional_get.rb:25:in `call'
 - rack (2.0.3) lib/rack/head.rb:12:in `call'
 - actionpack (5.1.4) lib/action_dispatch/middleware/callbacks.rb:26:in `block in call'
 - activesupport (5.1.4) lib/active_support/callbacks.rb:97:in `run_callbacks'
 - actionpack (5.1.4) lib/action_dispatch/middleware/callbacks.rb:24:in `call'
 - better_errors (2.3.0) lib/better_errors/middleware.rb:84:in `protected_app_call'
 - better_errors (2.3.0) lib/better_errors/middleware.rb:79:in `better_errors_call'
 - better_errors (2.3.0) lib/better_errors/middleware.rb:57:in `call'
 - actionpack (5.1.4) lib/action_dispatch/middleware/debug_exceptions.rb:59:in `call'
 - actionpack (5.1.4) lib/action_dispatch/middleware/show_exceptions.rb:31:in `call'
 - railties (5.1.4) lib/rails/rack/logger.rb:36:in `call_app'
 - railties (5.1.4) lib/rails/rack/logger.rb:24:in `block in call'
 - activesupport (5.1.4) lib/active_support/tagged_logging.rb:69:in `block in tagged'
 - activesupport (5.1.4) lib/active_support/tagged_logging.rb:26:in `tagged'
 - activesupport (5.1.4) lib/active_support/tagged_logging.rb:69:in `tagged'
 - railties (5.1.4) lib/rails/rack/logger.rb:24:in `call'
 - actionpack (5.1.4) lib/action_dispatch/middleware/remote_ip.rb:79:in `call'
 - actionpack (5.1.4) lib/action_dispatch/middleware/request_id.rb:25:in `call'
 - rack (2.0.3) lib/rack/runtime.rb:22:in `call'
 - activesupport (5.1.4) lib/active_support/cache/strategy/local_cache_middleware.rb:27:in `call'
 - actionpack (5.1.4) lib/action_dispatch/middleware/executor.rb:12:in `call'
 - actionpack (5.1.4) lib/action_dispatch/middleware/static.rb:125:in `call'
 - rack (2.0.3) lib/rack/sendfile.rb:111:in `call'
 - rack-cors (1.0.2) lib/rack/cors.rb:97:in `call'
 - railties (5.1.4) lib/rails/engine.rb:522:in `call'
 - rack-test (0.7.0) lib/rack/mock_session.rb:30:in `request'
 - rack-test (0.7.0) lib/rack/test.rb:249:in `process_request'
 - rack-test (0.7.0) lib/rack/test.rb:125:in `request'
 - actionpack (5.1.4) lib/action_dispatch/testing/integration.rb:261:in `process'
 - actionpack (5.1.4) lib/action_dispatch/testing/integration.rb:16:in `get'
 - actionpack (5.1.4) lib/action_dispatch/testing/integration.rb:348:in `block (2 levels) in <module:Runner>'

I did some inspection of some of that code and there are a few things that seemed out of place to me.

ActionDispatch::Request:Session.create includes a previous session.

def self.create(store, req, default_options)
  session_was = find req
  session     = Request::Session.new(store, req)
  session.merge! session_was if session_was

  set(req, session)
  Options.set(req, Request::Session::Options.new(store, default_options))
  session
end

If I break here I see that session_was is set, and it is attempted to merge! into the new session. This is where load! is eventually called on the new session. As an experiment I tried forcing session_was to nil. This caused the test to succeed! I imagine because it didn't try to merge in the old session, but was the old session even necessary?

The newly created session in point 1 fails at load! because it doesn't have a rack.session.options header. I'm not sure why this is, but it is added in the Options.set line.
If I remove the sign_in call in the test, I get to my controller code successfully.

On the whole this feels like some sort of user error on my part, but I can't seem to figure out just what the problem might be.

The text was updated successfully, but these errors were encountered:

tegon · 2017-12-13T00:56:00Z

Hello @RocketPuppy, thanks for your report.
Indeed, the code where you're getting the error is not even from devise, but from action_dispatch.
I recommend that you post this question on StackOverflow. There a wider community will be able to help you. We try to keep only issues here.

Thank you!

mrstif · 2018-03-15T19:35:35Z

@RocketPuppy did you end up figuring out what this was?

I backtracked through the code and realised that this is happening since Warden::Manager places a previous session in the env... though I cannot understand why:

env[Rack::RACK_SESSION]
{"warden.user.user.session"=>{"last_request_at"=>1521141461}}

this actually doesn't happen in normal Rails mode...

RocketPuppy · 2018-03-15T20:28:42Z

I did not. I hacked my way around it by writing a test helper that intercepts the rails API requests.

mrstif · 2018-03-26T13:21:10Z

@RocketPuppy I seem to have found the reason why this to be happening.

Apparently, in rails-api mode, the ActionDispatch::Cookies and ActionDispatch::Session::CookieStore middlewares are inserted in the end of the middleware stack, which doesn't occur in normal Rails mode.

Due to this, those middlewares are included after Warden::Manager which messes up something in request specs...

Therefore, this solved it:

Rails.application.config.middleware.insert_before Warden::Manager, ActionDispatch::Cookies
Rails.application.config.middleware.insert_before Warden::Manager, ActionDispatch::Session::CookieStore

Hope it helps

tegon · 2018-03-28T12:11:56Z

Yeah, warden needs to run after a session middleware in order to work - which I guess it doesn't make much sense in an API application since they usually don't keep a session.
But anyway, thanks for digging into this!

emersonthis · 2019-03-08T04:29:19Z

I recently chased my tail for a while trying to figure this out. I am using Devise with Rails in api mode. In my case, the API is powering a public single page app, so I'm using a standard session authentication instead of tokens.

I wonder if the gem could somehow incorporate the fix above so that it works smoothly in API mode. Or if that's undesirable, how would you feel about a PR to mention this "gotcha" in the README?

tegon · 2019-03-13T13:18:08Z

I'm not sure if Devise as an engine should assume an application's middleware chain structure in order to achieve this. Of course, we can know for how a stock Rails application looks like, but we can't know how other applications look like - i.e. they will have custom middlewares many times.
About adding this in the README, I think it's great. I'm thinking in a new section that includes instructions or limitations on API mode so that we can later add other known ones. WDYT?

emersonthis · 2019-03-13T15:42:14Z

That makes complete sense to me. Shall I open that PR?

…

On Wed, Mar 13, 2019 at 6:18 AM Leonardo Tegon ***@***.***> wrote: I'm not sure if Devise as an engine should assume an application's middleware chain structure in order to achieve this. Of course, we can know for how a stock Rails application looks like, but we can't know how other applications look like - i.e. they will have custom middlewares many times. About adding this in the README, I think it's great. I'm thinking in a new section that includes instructions or limitations on API mode so that we can later add other known ones. WDYT? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#4696 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABpei8yrdSYM5e9j1Rg2Uycgob0uixCAks5vWPqfgaJpZM4QVEMg> .

tegon · 2019-03-13T16:26:51Z

Yeah, that would be great! Thanks!

I found an entry in the Devise README which gave notice of exactly this error happening when the Warden middleware is initialized before the session middleware. The solution was to explicitly define the proper order in the test environment. See https://github.com/heartcombo/devise#testing and heartcombo/devise#4696.

tegon closed this as completed Dec 13, 2017

emersonthis mentioned this issue Mar 14, 2019

Added mention of API mode complications to README #5041

Merged

YassineDM mentioned this issue Aug 2, 2019

Sign in and out a user in Request type specs tuto not working anymore when integrating OmniAuth into Rails API #5105

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

sign_in IntegrationHelper produces errors when trying to load a session #4696

sign_in IntegrationHelper produces errors when trying to load a session #4696

RocketPuppy commented Nov 7, 2017 •

edited

tegon commented Dec 13, 2017

mrstif commented Mar 15, 2018 •

edited

RocketPuppy commented Mar 15, 2018

mrstif commented Mar 26, 2018 •

edited

tegon commented Mar 28, 2018

emersonthis commented Mar 8, 2019

tegon commented Mar 13, 2019

emersonthis commented Mar 13, 2019 via email

tegon commented Mar 13, 2019

sign_in IntegrationHelper produces errors when trying to load a session #4696

sign_in IntegrationHelper produces errors when trying to load a session #4696

Comments

RocketPuppy commented Nov 7, 2017 • edited

tegon commented Dec 13, 2017

mrstif commented Mar 15, 2018 • edited

RocketPuppy commented Mar 15, 2018

mrstif commented Mar 26, 2018 • edited

tegon commented Mar 28, 2018

emersonthis commented Mar 8, 2019

tegon commented Mar 13, 2019

emersonthis commented Mar 13, 2019 via email

tegon commented Mar 13, 2019

RocketPuppy commented Nov 7, 2017 •

edited

mrstif commented Mar 15, 2018 •

edited

mrstif commented Mar 26, 2018 •

edited