Feeds requiring cookie support

[sanjayankur31]

Hello,

Thanks for pluto, we moved the neuroscience planets over from venus a few days ago:

https://github.com/neurofedora/planet-neuroscience
https://github.com/neurofedora/planet-neuroscientists

For the neuroscience planet, which collects feeds from research journals, we're seeing errors which seem to be related to cookies:

[debug] fetch - get(_response) src: https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827
[debug] using direct net http access; no proxy configured
[debug] GET /search.rss?search-within=Journal&facet-journal-id=10827 uri=https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827, redirect_limit=5
[info] found cache entry for >https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827<
[debug] 302 Found location=https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827&client_id=springerlink&response_type=cookie
[debug] GET /authorize?redirect_uri=https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827&client_id=springerlink&response_type=cookie uri=https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827&client_id=springerlink&response_type=cookie, redirect_limit=4
[debug] 302 Found location=https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fsearch.rss%3Fsearch-within%3DJournal&code=5a9cf68b-e1ee-417a-8ca8-c6c202202caf
[debug] GET /transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fsearch.rss%3Fsearch-within%3DJournal&code=5a9cf68b-e1ee-417a-8ca8-c6c202202caf uri=https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fsearch.rss%3Fsearch-within%3DJournal&code=5a9cf68b-e1ee-417a-8ca8-c6c202202caf, redirect_limit=3
[debug] 302 Found location=https://link.springer.com/search.rss?search-within=Journal&error=cookies_not_supported&code=5a9cf68b-e1ee-417a-8ca8-c6c202202caf
[debug] GET /search.rss?search-within=Journal&error=cookies_not_supported&code=5a9cf68b-e1ee-417a-8ca8-c6c202202caf uri=https://link.springer.com/search.rss?search-within=Journal&error=cookies_not_supported&code=5a9cf68b-e1ee-417a-8ca8-c6c202202caf, redirect_limit=2
[debug] 302 Found location=null?idp_error=cookies_not_supported
[debug] url relative; try to make it absolute
[debug] GET /null?idp_error=cookies_not_supported uri=https://link.springer.com/null?idp_error=cookies_not_supported, redirect_limit=1
[debug] 302 Found location=https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie
[debug] GET /authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie uri=https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie, redirect_limit=0
[debug] 302 Found location=https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=e99fb052-66cc-4e41-ba89-9072db17e558

*** error: HTTP redirect too deep

/home/asinha/.local/share/gem/ruby/gems/fetcher-0.4.5/lib/fetcher/worker.rb:166:in `get_response': HTTP redirect too deep (ArgumentError)
        from /home/asinha/.local/share/gem/ruby/gems/fetcher-0.4.5/lib/fetcher/worker.rb:56:in `get'
        from /home/asinha/.local/share/gem/ruby/gems/pluto-feedfetcher-0.1.6/lib/pluto/feedfetcher/cond_get_with_cache.rb:34:in `fetch'
        from /home/asinha/.local/share/gem/ruby/gems/pluto-update-1.6.4/lib/pluto/update/feed_refresher.rb:57:in `refresh_feed_worker'
        from /home/asinha/.local/share/gem/ruby/gems/pluto-update-1.6.4/lib/pluto/update/feed_refresher.rb:49:in `block in refresh_feeds_for'
        from /home/asinha/.local/share/gem/ruby/gems/activerecord-6.1.4/lib/active_record/relation/delegation.rb:88:in `each'
        from /home/asinha/.local/share/gem/ruby/gems/activerecord-6.1.4/lib/active_record/relation/delegation.rb:88:in `each'
        from /home/asinha/.local/share/gem/ruby/gems/pluto-update-1.6.4/lib/pluto/update/feed_refresher.rb:48:in `refresh_feeds_for'
        from /home/asinha/.local/share/gem/ruby/gems/pluto-1.3.4/lib/pluto/cli/updater.rb:51:in `update_for'
        from /home/asinha/.local/share/gem/ruby/gems/pluto-1.3.4/lib/pluto/cli/updater.rb:31:in `run'
        from /home/asinha/.local/share/gem/ruby/gems/pluto-1.3.4/lib/pluto/cli/main.rb:289:in `block (3 levels) in <top (required)>'
        from /home/asinha/.local/share/gem/ruby/gems/pluto-1.3.4/lib/pluto/cli/main.rb:270:in `each'
        from /home/asinha/.local/share/gem/ruby/gems/pluto-1.3.4/lib/pluto/cli/main.rb:270:in `block (2 levels) in <top (required)>'
        from /home/asinha/.local/share/gem/ruby/gems/gli-2.20.1/lib/gli/command_support.rb:131:in `execute'
        from /home/asinha/.local/share/gem/ruby/gems/gli-2.20.1/lib/gli/app_support.rb:296:in `block in call_command'
        from /home/asinha/.local/share/gem/ruby/gems/gli-2.20.1/lib/gli/app_support.rb:309:in `call_command'
        from /home/asinha/.local/share/gem/ruby/gems/gli-2.20.1/lib/gli/app_support.rb:83:in `run'
        from /home/asinha/.local/share/gem/ruby/gems/pluto-1.3.4/lib/pluto/cli/main.rb:390:in `<top (required)>'
        from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require'
        from <internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require'
        from /home/asinha/.local/share/gem/ruby/gems/activesupport-6.1.4/lib/active_support/dependencies.rb:332:in `block in require'
        from /home/asinha/.local/share/gem/ruby/gems/activesupport-6.1.4/lib/active_support/dependencies.rb:299:in `load_dependency'
        from /home/asinha/.local/share/gem/ruby/gems/activesupport-6.1.4/lib/active_support/dependencies.rb:332:in `require'
        from /home/asinha/.local/share/gem/ruby/gems/pluto-1.3.4/lib/pluto.rb:24:in `main'
        from /home/asinha/.local/share/gem/ruby/gems/pluto-1.3.4/bin/pluto:5:in `<top (required)>'
        from /home/asinha/bin/pluto:23:in `load'
        from /home/asinha/bin/pluto:23:in `<main>'

https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827 works fine in a browser, but not on pluto---and it looks like they keep redirecting which causes pluto to error out eventually.

Is there a way to use these feeds with pluto?

[geraldb]

Thanks for highlighting the neuroscience planets.

The http library used is ruby's standard Net::HTTP with a little convenience wrapper (for handling redirections and caching but not cookies), See https://github.com/rubycoco/webclient/tree/master/fetcher

If you have pluto installed locally you should also have the fetcher gem install which includes a debug command line tool called fetch. Try:

$ fetch -v "https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827"

That should give you the same result, please what results you get (theoratically it's the same but "isolated" for easy testing. We can take it from there. You might check what is the spring policy on feeds and if you need a token or something.

Ok I tried:

 fetch -v "https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827"

That gets redirected to https://idp.springer.com/authorize?redirect_uri=[url] - not sure what to do here.

Not sure if you can do ruby scripts - if you can - than use the standard Net:HTTP library and make a script with a successful request with cookie and I am happy to patch / update the fetcher based on your script. This cheatsheet might help to get your started https://yukimotopress.github.io/http.

[sanjayankur31]

Thanks for looking into it @geraldb . I've dug in a little more. Using something like wget works fine:

$ wget "https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827" -O rss.xml
--2021-08-05 17:02:17--  https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827
Resolving link.springer.com (link.springer.com)... 151.101.192.95, 151.101.128.95, 151.101.0.95, ...
Connecting to link.springer.com (link.springer.com)|151.101.192.95|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827&client_id=springerlink&response_type=cookie [following]
--2021-08-05 17:02:17--  https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827&client_id=springerlink&response_type=cookie
Resolving idp.springer.com (idp.springer.com)... 151.101.64.95, 151.101.192.95, 151.101.0.95, ...
Connecting to idp.springer.com (idp.springer.com)|151.101.64.95|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fsearch.rss%3Fsearch-within%3DJournal&code=92e2f50e-58c9-4c0e-88d2-5da736592777 [following]
--2021-08-05 17:02:17--  https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fsearch.rss%3Fsearch-within%3DJournal&code=92e2f50e-58c9-4c0e-88d2-5da736592777
Reusing existing connection to idp.springer.com:443.
HTTP request sent, awaiting response... 302 Found
Location: https://link.springer.com/search.rss?search-within=Journal [following]
--2021-08-05 17:02:17--  https://link.springer.com/search.rss?search-within=Journal
Connecting to link.springer.com (link.springer.com)|151.101.192.95|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5575 (5.4K) [text/xml]
Saving to: ‘rss.xml’

rss.xml                              100%[====================================================================>]   5.44K  --.-KB/s    in 0.001s  

2021-08-05 17:02:18 (5.57 MB/s) - ‘rss.xml’ saved [5575/5575]

However, if I stop wget from dealing with cookies, I get the same redirection loop:

wget --no-cookies "https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827" -O rss.xml
--2021-08-05 17:09:44--  https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827
Resolving link.springer.com (link.springer.com)... 151.101.64.95, 151.101.128.95, 151.101.192.95, ...
Connecting to link.springer.com (link.springer.com)|151.101.64.95|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827&client_id=springerlink&response_type=cookie [following]
--2021-08-05 17:09:44--  https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827&client_id=springerlink&response_type=cookie
Resolving idp.springer.com (idp.springer.com)... 151.101.0.95, 151.101.192.95, 151.101.128.95, ...
Connecting to idp.springer.com (idp.springer.com)|151.101.0.95|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fsearch.rss%3Fsearch-within%3DJournal&code=a349f8e1-a7aa-4fc7-812c-91b3e7f5fa56 [following]
--2021-08-05 17:09:44--  https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fsearch.rss%3Fsearch-within%3DJournal&code=a349f8e1-a7aa-4fc7-812c-91b3e7f5fa56
Reusing existing connection to idp.springer.com:443.
HTTP request sent, awaiting response... 302 Found
Location: https://link.springer.com/search.rss?search-within=Journal&error=cookies_not_supported&code=a349f8e1-a7aa-4fc7-812c-91b3e7f5fa56 [following]
--2021-08-05 17:09:44--  https://link.springer.com/search.rss?search-within=Journal&error=cookies_not_supported&code=a349f8e1-a7aa-4fc7-812c-91b3e7f5fa56
Connecting to link.springer.com (link.springer.com)|151.101.64.95|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: null?idp_error=cookies_not_supported [following]
--2021-08-05 17:09:44--  https://link.springer.com/null?idp_error=cookies_not_supported
Reusing existing connection to link.springer.com:443.
HTTP request sent, awaiting response... 302 Found
Location: https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie [following]
--2021-08-05 17:09:44--  https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie
Connecting to idp.springer.com (idp.springer.com)|151.101.0.95|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=6c7a1514-6880-4f86-af62-a8257a544bb2 [following]
--2021-08-05 17:09:45--  https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=6c7a1514-6880-4f86-af62-a8257a544bb2
Reusing existing connection to idp.springer.com:443.
HTTP request sent, awaiting response... 302 Found
Location: https://link.springer.com/null?idp_error=cookies_not_supported&error=cookies_not_supported&code=6c7a1514-6880-4f86-af62-a8257a544bb2 [following]
--2021-08-05 17:09:45--  https://link.springer.com/null?idp_error=cookies_not_supported&error=cookies_not_supported&code=6c7a1514-6880-4f86-af62-a8257a544bb2
Connecting to link.springer.com (link.springer.com)|151.101.64.95|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: /null?idp_error=cookies_not_supported [following]
--2021-08-05 17:09:45--  https://link.springer.com/null?idp_error=cookies_not_supported
Reusing existing connection to link.springer.com:443.
HTTP request sent, awaiting response... 302 Found
Location: https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie [following]
--2021-08-05 17:09:45--  https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie
Connecting to idp.springer.com (idp.springer.com)|151.101.0.95|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=0fd7710a-78b4-41b7-84e1-c6154f507d4e [following]
--2021-08-05 17:09:45--  https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=0fd7710a-78b4-41b7-84e1-c6154f507d4e
Reusing existing connection to idp.springer.com:443.
HTTP request sent, awaiting response... 302 Found
Location: https://link.springer.com/null?idp_error=cookies_not_supported&error=cookies_not_supported&code=0fd7710a-78b4-41b7-84e1-c6154f507d4e [following]
--2021-08-05 17:09:45--  https://link.springer.com/null?idp_error=cookies_not_supported&error=cookies_not_supported&code=0fd7710a-78b4-41b7-84e1-c6154f507d4e
Connecting to link.springer.com (link.springer.com)|151.101.64.95|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: /null?idp_error=cookies_not_supported [following]
--2021-08-05 17:09:45--  https://link.springer.com/null?idp_error=cookies_not_supported
Reusing existing connection to link.springer.com:443.
HTTP request sent, awaiting response... 302 Found
Location: https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie [following]
--2021-08-05 17:09:45--  https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie
Connecting to idp.springer.com (idp.springer.com)|151.101.0.95|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=4fe6e4ba-89d6-4a67-a9dc-46179249bf29 [following]
--2021-08-05 17:09:45--  https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=4fe6e4ba-89d6-4a67-a9dc-46179249bf29
Reusing existing connection to idp.springer.com:443.
HTTP request sent, awaiting response... 302 Found
Location: https://link.springer.com/null?idp_error=cookies_not_supported&error=cookies_not_supported&code=4fe6e4ba-89d6-4a67-a9dc-46179249bf29 [following]
--2021-08-05 17:09:45--  https://link.springer.com/null?idp_error=cookies_not_supported&error=cookies_not_supported&code=4fe6e4ba-89d6-4a67-a9dc-46179249bf29
Connecting to link.springer.com (link.springer.com)|151.101.64.95|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: /null?idp_error=cookies_not_supported [following]
--2021-08-05 17:09:45--  https://link.springer.com/null?idp_error=cookies_not_supported
Reusing existing connection to link.springer.com:443.
HTTP request sent, awaiting response... 302 Found
Location: https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie [following]
--2021-08-05 17:09:45--  https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie
Connecting to idp.springer.com (idp.springer.com)|151.101.0.95|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=d57a8274-6c47-4284-a12f-bcdd77cf1613 [following]
--2021-08-05 17:09:45--  https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=d57a8274-6c47-4284-a12f-bcdd77cf1613
Reusing existing connection to idp.springer.com:443.
HTTP request sent, awaiting response... 302 Found
Location: https://link.springer.com/null?idp_error=cookies_not_supported&error=cookies_not_supported&code=d57a8274-6c47-4284-a12f-bcdd77cf1613 [following]
--2021-08-05 17:09:45--  https://link.springer.com/null?idp_error=cookies_not_supported&error=cookies_not_supported&code=d57a8274-6c47-4284-a12f-bcdd77cf1613
Connecting to link.springer.com (link.springer.com)|151.101.64.95|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: /null?idp_error=cookies_not_supported [following]
--2021-08-05 17:09:45--  https://link.springer.com/null?idp_error=cookies_not_supported
Reusing existing connection to link.springer.com:443.
HTTP request sent, awaiting response... 302 Found
Location: https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie [following]
20 redirections exceeded.

So I guess the fix/workaround here would be to get fetcher etc. to support cookies. I don't do a lot of ruby, but I'll see if someone from the team can find the time to work on this.

[geraldb]

Thanks for the wget analysis - that makes it look like as you say it requires to pass along the set-cookie on the redirect. Not sure if wget also has an option to show the request and response HTTP headers - it should be something like set-cookie in the first HTTP equest and cookie in the second HTTP response.

I try to put together a test script in about a week - if someone in your team can try out a request with plain ruby Net::HTTP that be great or even python if you set / manage the cookies yourself.

[sanjayankur31]

Yes, here are the outputs in debug mode which includes the headers:

With cookies:

wget -d "https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827" -O rss.xml
Setting --output-document (outputdocument) to rss.xml
DEBUG output created by Wget 1.21.1 on linux-gnu.

Reading HSTS entries from /home/asinha/.wget-hsts
URI encoding = ‘UTF-8’
--2021-08-05 18:36:29--  https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827
Certificates loaded: 143
Resolving link.springer.com (link.springer.com)... 151.101.128.95, 151.101.64.95, 151.101.192.95, ...
Caching link.springer.com => 151.101.128.95 151.101.64.95 151.101.192.95 151.101.0.95
Connecting to link.springer.com (link.springer.com)|151.101.128.95|:443... connected.
Created socket 4.
Releasing 0x000055ad6aa05e50 (new refcount 1).

---request begin---
GET /search.rss?search-within=Journal&facet-journal-id=10827 HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: link.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: max-age=0
Expires: Thu, 05 Aug 2021 17:36:29 GMT
Location: https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827&client_id=springerlink&response_type=cookie
Server: Oscar Platform 0.314.0
X-Environment: live
X-Frame-Options: DENY
X-Origin-Server: ac110a5e-efe8-41b5-530c-d27e
X-Vcap-Request-Id: 66075e3c-41d5-4522-4e99-d382249cdc77
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:36:29 GMT
Age: 0
X-Served-By: cache-d3fae307-internal, cache-lhr7338-LHR
X-Cache: MISS, MISS
X-Cache-Hits: 0
X-Timer: S1628184990.530510,VS0,VE41

---response end---
302 Found
Registered socket 4 for persistent reuse.
Location: https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827&client_id=springerlink&response_type=cookie [following]
] done.
URI content encoding = None
--2021-08-05 18:36:29--  https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827&client_id=springerlink&response_type=cookie
Resolving idp.springer.com (idp.springer.com)... 151.101.128.95, 151.101.0.95, 151.101.192.95, ...
Caching idp.springer.com => 151.101.128.95 151.101.0.95 151.101.192.95 151.101.64.95
Connecting to idp.springer.com (idp.springer.com)|151.101.128.95|:443... connected.
Created socket 5.
Releasing 0x000055ad6aa1f690 (new refcount 1).

---request begin---
GET /authorize?redirect_uri=https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827&client_id=springerlink&response_type=cookie HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: idp.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Expires: 0
Location: https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fsearch.rss%3Fsearch-within%3DJournal&code=b1403774-9120-4984-8449-04663e89505a
Pragma: no-cache
Set-Cookie: idp_session=sVERSION_1b8226973-81df-4261-b453-7b221ddf56fd; Domain=.springer.com; Path=/; Secure; HttpOnly
Set-Cookie: idp_session_http=hVERSION_1598a3ad7-1ef8-49bf-9dd1-084eb935b38d; Domain=.springer.com; Path=/; HttpOnly
Set-Cookie: idp_marker=9e2e1951-e2ad-4669-8419-f747eefded20; Domain=.springer.com; Path=/; Max-Age=315360000; HttpOnly
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-B3-Spanid: a885e10befbf1bae
X-B3-Traceid: a885e10befbf1bae
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Vcap-Request-Id: 4aa7cb6b-5b61-4224-6ab6-cd61d214009e
X-Xss-Protection: 1; mode=block
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:36:29 GMT
X-Served-By: cache-lcy19262-LCY
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1628184990.654500,VS0,VE21
Vary: x-forwarded-proto

---response end---
302 Found
cdm: 1

Stored cookie springer.com -1 (ANY) / <session> <secure> [expiry none] idp_session sVERSION_1b8226973-81df-4261-b453-7b221ddf56fd
cdm: 1

Stored cookie springer.com -1 (ANY) / <session> <insecure> [expiry none] idp_session_http hVERSION_1598a3ad7-1ef8-49bf-9dd1-084eb935b38d
cdm: 1

Stored cookie springer.com -1 (ANY) / <permanent> <insecure> [expiry 2031-08-03 18:36:29] idp_marker 9e2e1951-e2ad-4669-8419-f747eefded20
Disabling further reuse of socket 4.
Registered socket 5 for persistent reuse.
Parsed Strict-Transport-Security max-age = 31536000, includeSubDomains = true
Updated HSTS host: idp.springer.com:443 (max-age: 31536000, includeSubdomains: true)
Location: https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fsearch.rss%3Fsearch-within%3DJournal&code=b1403774-9120-4984-8449-04663e89505a [following]
] done.
URI content encoding = None
--2021-08-05 18:36:29--  https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fsearch.rss%3Fsearch-within%3DJournal&code=b1403774-9120-4984-8449-04663e89505a
Reusing existing connection to idp.springer.com:443.
Reusing fd 5.

---request begin---
GET /transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fsearch.rss%3Fsearch-within%3DJournal&code=b1403774-9120-4984-8449-04663e89505a HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: idp.springer.com
Connection: Keep-Alive
Cookie: idp_marker=9e2e1951-e2ad-4669-8419-f747eefded20; idp_session=sVERSION_1b8226973-81df-4261-b453-7b221ddf56fd; idp_session_http=hVERSION_1598a3ad7-1ef8-49bf-9dd1-084eb935b38d

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Language: en-US
Expires: 0
Location: https://link.springer.com/search.rss?search-within=Journal
Pragma: no-cache
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-B3-Spanid: 1ab7f4e10b76e4e8
X-B3-Traceid: 1ab7f4e10b76e4e8
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Vcap-Request-Id: 964d5237-9319-48d9-43b6-076e8adcd874
X-Xss-Protection: 1; mode=block
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:36:29 GMT
X-Served-By: cache-lcy19262-LCY
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1628184990.704945,VS0,VE15
Vary: x-forwarded-proto

---response end---
302 Found
Parsed Strict-Transport-Security max-age = 31536000, includeSubDomains = true
Updated HSTS host: idp.springer.com:443 (max-age: 31536000, includeSubdomains: true)
Location: https://link.springer.com/search.rss?search-within=Journal [following]
] done.
URI content encoding = None
--2021-08-05 18:36:29--  https://link.springer.com/search.rss?search-within=Journal
Found link.springer.com in host_name_addresses_map (0x55ad6aa05e50)
Connecting to link.springer.com (link.springer.com)|151.101.128.95|:443... connected.
Created socket 4.
Releasing 0x000055ad6aa05e50 (new refcount 1).

---request begin---
GET /search.rss?search-within=Journal HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: link.springer.com
Connection: Keep-Alive
Cookie: idp_marker=9e2e1951-e2ad-4669-8419-f747eefded20; idp_session=sVERSION_1b8226973-81df-4261-b453-7b221ddf56fd; idp_session_http=hVERSION_1598a3ad7-1ef8-49bf-9dd1-084eb935b38d

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 5575
Cache-Control: max-age=86400
Content-Type: text/xml;charset=utf-8
Expires: Fri, 06 Aug 2021 13:31:03 GMT
Server: Oscar Platform 0.314.0
X-Environment: live
X-Frame-Options: DENY
X-Origin-Server: 9d449640-d9d5-40e8-460a-733f
X-Vcap-Request-Id: b6460770-cb63-49ee-44a2-61145196c410
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:36:29 GMT
Age: 14725
X-Served-By: cache-bc5ba314-internal, cache-lcy19249-LCY
X-Cache: HIT, HIT
X-Cache-Hits: 1
X-Timer: S1628184990.810559,VS0,VE1
Vary: Accept-Encoding

---response end---
200 OK
Disabling further reuse of socket 5.
Registered socket 4 for persistent reuse.
URI content encoding = ‘utf-8’
Length: 5575 (5.4K) [text/xml]
Saving to: ‘rss.xml’

rss.xml                              100%[====================================================================>]   5.44K  --.-KB/s    in 0.001s

2021-08-05 18:36:29 (10.3 MB/s) - ‘rss.xml’ saved [5575/5575]

Saving HSTS entries to /home/asinha/.wget-hsts

Without cookies:

wget -d --no-cookies "https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827" -O rss.xml
Setting --cookies (cookies) to 0
Setting --output-document (outputdocument) to rss.xml
DEBUG output created by Wget 1.21.1 on linux-gnu.

Reading HSTS entries from /home/asinha/.wget-hsts
URI encoding = ‘UTF-8’
--2021-08-05 18:35:26--  https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827
Certificates loaded: 143
Resolving link.springer.com (link.springer.com)... 151.101.64.95, 151.101.192.95, 151.101.128.95, ...
Caching link.springer.com => 151.101.64.95 151.101.192.95 151.101.128.95 151.101.0.95
Connecting to link.springer.com (link.springer.com)|151.101.64.95|:443... connected.
Created socket 4.
Releasing 0x000055f6ab678630 (new refcount 1).

---request begin---
GET /search.rss?search-within=Journal&facet-journal-id=10827 HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: link.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: max-age=0
Expires: Thu, 05 Aug 2021 17:35:26 GMT
Location: https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827&client_id=springerlink&response_type=cookie
Server: Oscar Platform 0.314.0
X-Environment: live
X-Frame-Options: DENY
X-Origin-Server: 1d728629-510b-47e9-4424-6d56
X-Vcap-Request-Id: 87180e25-265f-43c0-65c8-a793355ba5ae
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:35:26 GMT
Age: 0
X-Served-By: cache-d3fae307-internal, cache-lon4273-LON
X-Cache: MISS, MISS
X-Cache-Hits: 0
X-Timer: S1628184926.364757,VS0,VE41

---response end---
302 Found
Registered socket 4 for persistent reuse.
Location: https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827&client_id=springerlink&response_type=cookie [following]
] done.
URI content encoding = None
--2021-08-05 18:35:26--  https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827&client_id=springerlink&response_type=cookie
Resolving idp.springer.com (idp.springer.com)... 151.101.64.95, 151.101.128.95, 151.101.0.95, ...
Caching idp.springer.com => 151.101.64.95 151.101.128.95 151.101.0.95 151.101.192.95
Connecting to idp.springer.com (idp.springer.com)|151.101.64.95|:443... connected.
Created socket 5.
Releasing 0x000055f6ab4ed5e0 (new refcount 1).

---request begin---
GET /authorize?redirect_uri=https://link.springer.com/search.rss?search-within=Journal&facet-journal-id=10827&client_id=springerlink&response_type=cookie HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: idp.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Expires: 0
Location: https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fsearch.rss%3Fsearch-within%3DJournal&code=adad4bfe-aa93-4392-85a3-2ba7fd82029e
Pragma: no-cache
Set-Cookie: idp_session=sVERSION_1ff74b98a-59f3-4d61-97af-17c50460bc87; Domain=.springer.com; Path=/; Secure; HttpOnly
Set-Cookie: idp_session_http=hVERSION_1780d1832-5454-44b2-b374-330829b9261d; Domain=.springer.com; Path=/; HttpOnly
Set-Cookie: idp_marker=f3787ecb-dee9-4a29-bbf2-cbde6f286b9d; Domain=.springer.com; Path=/; Max-Age=315360000; HttpOnly
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-B3-Spanid: bfa5fe556a612af0
X-B3-Traceid: bfa5fe556a612af0
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Vcap-Request-Id: 1766e3fa-56dc-46b0-7b7a-059657dd1328
X-Xss-Protection: 1; mode=block
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:35:26 GMT
X-Served-By: cache-lcy19267-LCY
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1628184927.526769,VS0,VE22
Vary: x-forwarded-proto

---response end---
302 Found
Disabling further reuse of socket 4.
Registered socket 5 for persistent reuse.
Parsed Strict-Transport-Security max-age = 31536000, includeSubDomains = true
Updated HSTS host: idp.springer.com:443 (max-age: 31536000, includeSubdomains: true)
Location: https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fsearch.rss%3Fsearch-within%3DJournal&code=adad4bfe-aa93-4392-85a3-2ba7fd82029e [following]
] done.
URI content encoding = None
--2021-08-05 18:35:26--  https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fsearch.rss%3Fsearch-within%3DJournal&code=adad4bfe-aa93-4392-85a3-2ba7fd82029e
Reusing existing connection to idp.springer.com:443.
Reusing fd 5.

---request begin---
GET /transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fsearch.rss%3Fsearch-within%3DJournal&code=adad4bfe-aa93-4392-85a3-2ba7fd82029e HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: idp.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Language: en-US
Expires: 0
Location: https://link.springer.com/search.rss?search-within=Journal&error=cookies_not_supported&code=adad4bfe-aa93-4392-85a3-2ba7fd82029e
Pragma: no-cache
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-B3-Spanid: 5cb82cdaf5b6a9ee
X-B3-Traceid: 5cb82cdaf5b6a9ee
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Vcap-Request-Id: 0489ebfc-19d5-4d8d-6b6d-62b717241d5b
X-Xss-Protection: 1; mode=block
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:35:26 GMT
X-Served-By: cache-lcy19267-LCY
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1628184927.565377,VS0,VE15
Vary: x-forwarded-proto

---response end---
302 Found
Parsed Strict-Transport-Security max-age = 31536000, includeSubDomains = true
Updated HSTS host: idp.springer.com:443 (max-age: 31536000, includeSubdomains: true)
Location: https://link.springer.com/search.rss?search-within=Journal&error=cookies_not_supported&code=adad4bfe-aa93-4392-85a3-2ba7fd82029e [following]
] done.
URI content encoding = None
--2021-08-05 18:35:26--  https://link.springer.com/search.rss?search-within=Journal&error=cookies_not_supported&code=adad4bfe-aa93-4392-85a3-2ba7fd82029e
Found link.springer.com in host_name_addresses_map (0x55f6ab678630)
Connecting to link.springer.com (link.springer.com)|151.101.64.95|:443... connected.
Created socket 4.
Releasing 0x000055f6ab678630 (new refcount 1).

---request begin---
GET /search.rss?search-within=Journal&error=cookies_not_supported&code=adad4bfe-aa93-4392-85a3-2ba7fd82029e HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: link.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: max-age=0
Expires: Thu, 05 Aug 2021 17:35:26 GMT
Location: null?idp_error=cookies_not_supported
Server: Oscar Platform 0.314.0
X-Environment: live
X-Frame-Options: DENY
X-Origin-Server: ac110a5e-efe8-41b5-530c-d27e
X-Vcap-Request-Id: 7396b62f-e8f4-4e9f-4f57-4735b83cc9fe
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:35:26 GMT
Age: 0
X-Served-By: cache-ca8aaefd-internal, cache-lon4225-LON
X-Cache: MISS, MISS
X-Cache-Hits: 0
X-Timer: S1628184927.664845,VS0,VE33

---response end---
302 Found
Disabling further reuse of socket 5.
Registered socket 4 for persistent reuse.
Location: null?idp_error=cookies_not_supported [following]
] done.
URI content encoding = None
--2021-08-05 18:35:26--  https://link.springer.com/null?idp_error=cookies_not_supported
Reusing existing connection to link.springer.com:443.
Reusing fd 4.

---request begin---
GET /null?idp_error=cookies_not_supported HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: link.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: max-age=0
Expires: Thu, 05 Aug 2021 17:35:26 GMT
Location: https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie
Server: Oscar Platform 0.314.0
X-Environment: live
X-Frame-Options: DENY
X-Origin-Server: 6801ee30-740c-4869-47c4-0c2a
X-Vcap-Request-Id: 1b4f322a-5311-40b7-7008-b831c73611a5
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:35:26 GMT
Age: 0
X-Served-By: cache-bc5ba314-internal, cache-lon4225-LON
X-Cache: MISS, MISS
X-Cache-Hits: 0
X-Timer: S1628184927.706111,VS0,VE30

---response end---
302 Found
Location: https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie [following]
] done.
URI content encoding = None
--2021-08-05 18:35:26--  https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie
Found idp.springer.com in host_name_addresses_map (0x55f6ab4ed5e0)
Connecting to idp.springer.com (idp.springer.com)|151.101.64.95|:443... connected.
Created socket 5.
Releasing 0x000055f6ab4ed5e0 (new refcount 1).

---request begin---
GET /authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: idp.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Expires: 0
Location: https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=df2493cb-43bc-45d0-98f4-7ddfbf0353d1
Pragma: no-cache
Set-Cookie: idp_session=sVERSION_1ef34add7-80ea-4298-96b7-ceb372981d5c; Domain=.springer.com; Path=/; Secure; HttpOnly
Set-Cookie: idp_session_http=hVERSION_1d0cd3c7e-c05f-424c-b8cf-a19b42955251; Domain=.springer.com; Path=/; HttpOnly
Set-Cookie: idp_marker=a6015fa7-7f0d-4bba-9623-e1051ac7b6a1; Domain=.springer.com; Path=/; Max-Age=315360000; HttpOnly
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-B3-Spanid: bf00da04bcbe6db1
X-B3-Traceid: bf00da04bcbe6db1
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Vcap-Request-Id: 1c96df02-4ac0-458f-7755-18a5cee652cc
X-Xss-Protection: 1; mode=block
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:35:26 GMT
X-Served-By: cache-lcy19257-LCY
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1628184927.825436,VS0,VE21
Vary: x-forwarded-proto

---response end---
302 Found
Disabling further reuse of socket 4.
Registered socket 5 for persistent reuse.
Parsed Strict-Transport-Security max-age = 31536000, includeSubDomains = true
Updated HSTS host: idp.springer.com:443 (max-age: 31536000, includeSubdomains: true)
Location: https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=df2493cb-43bc-45d0-98f4-7ddfbf0353d1 [following]
] done.
URI content encoding = None
--2021-08-05 18:35:26--  https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=df2493cb-43bc-45d0-98f4-7ddfbf0353d1
Reusing existing connection to idp.springer.com:443.
Reusing fd 5.

---request begin---
GET /transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=df2493cb-43bc-45d0-98f4-7ddfbf0353d1 HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: idp.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Language: en-US
Expires: 0
Location: https://link.springer.com/null?idp_error=cookies_not_supported&error=cookies_not_supported&code=df2493cb-43bc-45d0-98f4-7ddfbf0353d1
Pragma: no-cache
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-B3-Spanid: 7e57a157a5495f59
X-B3-Traceid: 7e57a157a5495f59
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Vcap-Request-Id: 33a7f752-0c40-43c8-5946-1019504cceb1
X-Xss-Protection: 1; mode=block
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:35:26 GMT
X-Served-By: cache-lcy19257-LCY
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1628184927.858572,VS0,VE14
Vary: x-forwarded-proto

---response end---
302 Found
Parsed Strict-Transport-Security max-age = 31536000, includeSubDomains = true
Updated HSTS host: idp.springer.com:443 (max-age: 31536000, includeSubdomains: true)
Location: https://link.springer.com/null?idp_error=cookies_not_supported&error=cookies_not_supported&code=df2493cb-43bc-45d0-98f4-7ddfbf0353d1 [following]
] done.
URI content encoding = None
--2021-08-05 18:35:26--  https://link.springer.com/null?idp_error=cookies_not_supported&error=cookies_not_supported&code=df2493cb-43bc-45d0-98f4-7ddfbf0353d1
Found link.springer.com in host_name_addresses_map (0x55f6ab678630)
Connecting to link.springer.com (link.springer.com)|151.101.64.95|:443... connected.
Created socket 4.
Releasing 0x000055f6ab678630 (new refcount 1).

---request begin---
GET /null?idp_error=cookies_not_supported&error=cookies_not_supported&code=df2493cb-43bc-45d0-98f4-7ddfbf0353d1 HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: link.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: max-age=0
Expires: Thu, 05 Aug 2021 17:35:26 GMT
Location: /null?idp_error=cookies_not_supported
Server: Oscar Platform 0.314.0
X-Environment: live
X-Frame-Options: DENY
X-Origin-Server: 16968c40-cff2-4c43-50b9-eac7
X-Vcap-Request-Id: 1eba5e2b-f9e9-416c-5932-93066791e0be
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:35:26 GMT
Age: 0
X-Served-By: cache-ca8aaefd-internal, cache-lon4260-LON
X-Cache: MISS, MISS
X-Cache-Hits: 0
X-Timer: S1628184927.925114,VS0,VE18

---response end---
302 Found
Disabling further reuse of socket 5.
Registered socket 4 for persistent reuse.
Location: /null?idp_error=cookies_not_supported [following]
] done.
URI content encoding = None
--2021-08-05 18:35:26--  https://link.springer.com/null?idp_error=cookies_not_supported
Reusing existing connection to link.springer.com:443.
Reusing fd 4.

---request begin---
GET /null?idp_error=cookies_not_supported HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: link.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: max-age=0
Expires: Thu, 05 Aug 2021 17:35:26 GMT
Location: https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie
Server: Oscar Platform 0.314.0
X-Environment: live
X-Frame-Options: DENY
X-Origin-Server: 6801ee30-740c-4869-47c4-0c2a
X-Vcap-Request-Id: 6a5984b6-9ac8-4c0a-44e1-4fd2d063d524
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:35:26 GMT
Age: 0
X-Served-By: cache-31508b91-internal, cache-lon4260-LON
X-Cache: MISS, MISS
X-Cache-Hits: 0
X-Timer: S1628184927.955100,VS0,VE31

---response end---
302 Found
Location: https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie [following]
] done.
URI content encoding = None
--2021-08-05 18:35:26--  https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie
Found idp.springer.com in host_name_addresses_map (0x55f6ab4ed5e0)
Connecting to idp.springer.com (idp.springer.com)|151.101.64.95|:443... connected.
Created socket 5.
Releasing 0x000055f6ab4ed5e0 (new refcount 1).

---request begin---
GET /authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: idp.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Expires: 0
Location: https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=a91fc1f4-57da-459e-8c1b-c09091c74be5
Pragma: no-cache
Set-Cookie: idp_session=sVERSION_10a015bf9-1503-457b-a7ee-632ba4783f79; Domain=.springer.com; Path=/; Secure; HttpOnly
Set-Cookie: idp_session_http=hVERSION_1f18f034c-cec0-4c7c-9291-e8a8277cdb00; Domain=.springer.com; Path=/; HttpOnly
Set-Cookie: idp_marker=c444055d-4b0c-4faf-a193-4fa31b7bbea6; Domain=.springer.com; Path=/; Max-Age=315360000; HttpOnly
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-B3-Spanid: b54cecb7daa783d9
X-B3-Traceid: b54cecb7daa783d9
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Vcap-Request-Id: 37ab8877-fc3d-4bfb-54bb-f2dbc7988d08
X-Xss-Protection: 1; mode=block
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:35:27 GMT
X-Served-By: cache-lhr7349-LHR
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1628184927.057954,VS0,VE19
Vary: x-forwarded-proto

---response end---
302 Found
Disabling further reuse of socket 4.
Registered socket 5 for persistent reuse.
Parsed Strict-Transport-Security max-age = 31536000, includeSubDomains = true
Updated HSTS host: idp.springer.com:443 (max-age: 31536000, includeSubdomains: true)
Location: https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=a91fc1f4-57da-459e-8c1b-c09091c74be5 [following]
] done.
URI content encoding = None
--2021-08-05 18:35:27--  https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=a91fc1f4-57da-459e-8c1b-c09091c74be5
Reusing existing connection to idp.springer.com:443.
Reusing fd 5.

---request begin---
GET /transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=a91fc1f4-57da-459e-8c1b-c09091c74be5 HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: idp.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Language: en-US
Expires: 0
Location: https://link.springer.com/null?idp_error=cookies_not_supported&error=cookies_not_supported&code=a91fc1f4-57da-459e-8c1b-c09091c74be5
Pragma: no-cache
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-B3-Spanid: ec9342492442dc80
X-B3-Traceid: ec9342492442dc80
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Vcap-Request-Id: 846ae1e8-5626-49db-67ae-75e01dcff159
X-Xss-Protection: 1; mode=block
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:35:27 GMT
X-Served-By: cache-lhr7349-LHR
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1628184927.097059,VS0,VE15
Vary: x-forwarded-proto

---response end---
302 Found
Parsed Strict-Transport-Security max-age = 31536000, includeSubDomains = true
Updated HSTS host: idp.springer.com:443 (max-age: 31536000, includeSubdomains: true)
Location: https://link.springer.com/null?idp_error=cookies_not_supported&error=cookies_not_supported&code=a91fc1f4-57da-459e-8c1b-c09091c74be5 [following]
] done.
URI content encoding = None
--2021-08-05 18:35:27--  https://link.springer.com/null?idp_error=cookies_not_supported&error=cookies_not_supported&code=a91fc1f4-57da-459e-8c1b-c09091c74be5
Found link.springer.com in host_name_addresses_map (0x55f6ab678630)
Connecting to link.springer.com (link.springer.com)|151.101.64.95|:443... connected.
Created socket 4.
Releasing 0x000055f6ab678630 (new refcount 1).

---request begin---
GET /null?idp_error=cookies_not_supported&error=cookies_not_supported&code=a91fc1f4-57da-459e-8c1b-c09091c74be5 HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: link.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: max-age=0
Expires: Thu, 05 Aug 2021 17:35:27 GMT
Location: /null?idp_error=cookies_not_supported
Server: Oscar Platform 0.314.0
X-Environment: live
X-Frame-Options: DENY
X-Origin-Server: 16968c40-cff2-4c43-50b9-eac7
X-Vcap-Request-Id: c0bc7100-9d79-4e9e-722d-8b8450e08901
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:35:27 GMT
Age: 0
X-Served-By: cache-e27548d0-internal, cache-lcy19251-LCY
X-Cache: MISS, MISS
X-Cache-Hits: 0
X-Timer: S1628184927.194741,VS0,VE17

---response end---
302 Found
Disabling further reuse of socket 5.
Registered socket 4 for persistent reuse.
Location: /null?idp_error=cookies_not_supported [following]
] done.
URI content encoding = None
--2021-08-05 18:35:27--  https://link.springer.com/null?idp_error=cookies_not_supported
Reusing existing connection to link.springer.com:443.
Reusing fd 4.

---request begin---
GET /null?idp_error=cookies_not_supported HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: link.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: max-age=0
Expires: Thu, 05 Aug 2021 17:35:27 GMT
Location: https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie
Server: Oscar Platform 0.314.0
X-Environment: live
X-Frame-Options: DENY
X-Origin-Server: ac110a5e-efe8-41b5-530c-d27e
X-Vcap-Request-Id: 6a1325c1-2702-4766-7e27-810190353d81
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:35:27 GMT
Age: 0
X-Served-By: cache-bc5ba314-internal, cache-lcy19251-LCY
X-Cache: MISS, MISS
X-Cache-Hits: 0
X-Timer: S1628184927.225391,VS0,VE33

---response end---
302 Found
Location: https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie [following]
] done.
URI content encoding = None
--2021-08-05 18:35:27--  https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie
Found idp.springer.com in host_name_addresses_map (0x55f6ab4ed5e0)
Connecting to idp.springer.com (idp.springer.com)|151.101.64.95|:443... connected.
Created socket 5.
Releasing 0x000055f6ab4ed5e0 (new refcount 1).

---request begin---
GET /authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: idp.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Expires: 0
Location: https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=96c2d0d7-3647-47c9-a82f-a917b87740dd
Pragma: no-cache
Set-Cookie: idp_session=sVERSION_1f4c7452d-ee45-4f71-8cc7-9e798b7872c3; Domain=.springer.com; Path=/; Secure; HttpOnly
Set-Cookie: idp_session_http=hVERSION_15d907e9b-04d9-4810-873b-ed428d7f9b5d; Domain=.springer.com; Path=/; HttpOnly
Set-Cookie: idp_marker=4ee8915e-ddf0-4f82-8197-52745294a6e1; Domain=.springer.com; Path=/; Max-Age=315360000; HttpOnly
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-B3-Spanid: 04243fe17a2fb0d3
X-B3-Traceid: 04243fe17a2fb0d3
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Vcap-Request-Id: b9ac3087-363d-4934-72bf-dc455822717f
X-Xss-Protection: 1; mode=block
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:35:27 GMT
X-Served-By: cache-lon4220-LON
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1628184927.335046,VS0,VE20
Vary: x-forwarded-proto

---response end---
302 Found
Disabling further reuse of socket 4.
Registered socket 5 for persistent reuse.
Parsed Strict-Transport-Security max-age = 31536000, includeSubDomains = true
Updated HSTS host: idp.springer.com:443 (max-age: 31536000, includeSubdomains: true)
Location: https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=96c2d0d7-3647-47c9-a82f-a917b87740dd [following]
] done.
URI content encoding = None
--2021-08-05 18:35:27--  https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=96c2d0d7-3647-47c9-a82f-a917b87740dd
Reusing existing connection to idp.springer.com:443.
Reusing fd 5.

---request begin---
GET /transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=96c2d0d7-3647-47c9-a82f-a917b87740dd HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: idp.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Language: en-US
Expires: 0
Location: https://link.springer.com/null?idp_error=cookies_not_supported&error=cookies_not_supported&code=96c2d0d7-3647-47c9-a82f-a917b87740dd
Pragma: no-cache
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-B3-Spanid: 663967c0f849e1d1
X-B3-Traceid: 663967c0f849e1d1
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Vcap-Request-Id: c2bc699b-1e4c-4a5b-4ec2-1b53ea22f6cf
X-Xss-Protection: 1; mode=block
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:35:27 GMT
X-Served-By: cache-lon4220-LON
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1628184927.365219,VS0,VE16
Vary: x-forwarded-proto

---response end---
302 Found
Parsed Strict-Transport-Security max-age = 31536000, includeSubDomains = true
Updated HSTS host: idp.springer.com:443 (max-age: 31536000, includeSubdomains: true)
Location: https://link.springer.com/null?idp_error=cookies_not_supported&error=cookies_not_supported&code=96c2d0d7-3647-47c9-a82f-a917b87740dd [following]
] done.
URI content encoding = None
--2021-08-05 18:35:27--  https://link.springer.com/null?idp_error=cookies_not_supported&error=cookies_not_supported&code=96c2d0d7-3647-47c9-a82f-a917b87740dd
Found link.springer.com in host_name_addresses_map (0x55f6ab678630)
Connecting to link.springer.com (link.springer.com)|151.101.64.95|:443... connected.
Created socket 4.
Releasing 0x000055f6ab678630 (new refcount 1).

---request begin---
GET /null?idp_error=cookies_not_supported&error=cookies_not_supported&code=96c2d0d7-3647-47c9-a82f-a917b87740dd HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: link.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: max-age=0
Expires: Thu, 05 Aug 2021 17:35:27 GMT
Location: /null?idp_error=cookies_not_supported
Server: Oscar Platform 0.314.0
X-Environment: live
X-Frame-Options: DENY
X-Origin-Server: 1d728629-510b-47e9-4424-6d56
X-Vcap-Request-Id: b3900020-ade2-46bf-41bb-d637b8cc42ea
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:35:27 GMT
Age: 0
X-Served-By: cache-b21f2443-internal, cache-lcy19233-LCY
X-Cache: MISS, MISS
X-Cache-Hits: 0
X-Timer: S1628184927.464852,VS0,VE18

---response end---
302 Found
Disabling further reuse of socket 5.
Registered socket 4 for persistent reuse.
Location: /null?idp_error=cookies_not_supported [following]
] done.
URI content encoding = None
--2021-08-05 18:35:27--  https://link.springer.com/null?idp_error=cookies_not_supported
Reusing existing connection to link.springer.com:443.
Reusing fd 4.

---request begin---
GET /null?idp_error=cookies_not_supported HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: link.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: max-age=0
Expires: Thu, 05 Aug 2021 17:35:27 GMT
Location: https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie
Server: Oscar Platform 0.314.0
X-Environment: live
X-Frame-Options: DENY
X-Origin-Server: a543f9b2-2464-424a-76f1-f1d5
X-Vcap-Request-Id: 645b3721-a34e-47d2-72c8-d89fa42efc55
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:35:27 GMT
Age: 0
X-Served-By: cache-bc5ba314-internal, cache-lcy19233-LCY
X-Cache: MISS, MISS
X-Cache-Hits: 0
X-Timer: S1628184928.505947,VS0,VE28

---response end---
302 Found
Location: https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie [following]
] done.
URI content encoding = None
--2021-08-05 18:35:27--  https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie
Found idp.springer.com in host_name_addresses_map (0x55f6ab4ed5e0)
Connecting to idp.springer.com (idp.springer.com)|151.101.64.95|:443... connected.
Created socket 5.
Releasing 0x000055f6ab4ed5e0 (new refcount 1).

---request begin---
GET /authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: idp.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Expires: 0
Location: https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=4fd6f2a7-d04b-407e-ae35-238137854ace
Pragma: no-cache
Set-Cookie: idp_session=sVERSION_160c8998c-531c-47f7-8ac2-11d7ac2e4a54; Domain=.springer.com; Path=/; Secure; HttpOnly
Set-Cookie: idp_session_http=hVERSION_12be5d291-43b2-4e2d-94c2-cbd181cec817; Domain=.springer.com; Path=/; HttpOnly
Set-Cookie: idp_marker=a3c38a34-5562-4ab1-b8e7-714d8dbaae80; Domain=.springer.com; Path=/; Max-Age=315360000; HttpOnly
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-B3-Spanid: 62adffa276dcff7c
X-B3-Traceid: 62adffa276dcff7c
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Vcap-Request-Id: 1a43b863-2919-4270-508c-b53b224900f9
X-Xss-Protection: 1; mode=block
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:35:27 GMT
X-Served-By: cache-lhr7325-LHR
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1628184928.606275,VS0,VE20
Vary: x-forwarded-proto

---response end---
302 Found
Disabling further reuse of socket 4.
Registered socket 5 for persistent reuse.
Parsed Strict-Transport-Security max-age = 31536000, includeSubDomains = true
Updated HSTS host: idp.springer.com:443 (max-age: 31536000, includeSubdomains: true)
Location: https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=4fd6f2a7-d04b-407e-ae35-238137854ace [following]
] done.
URI content encoding = None
--2021-08-05 18:35:27--  https://idp.springer.com/transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=4fd6f2a7-d04b-407e-ae35-238137854ace
Reusing existing connection to idp.springer.com:443.
Reusing fd 5.

---request begin---
GET /transit?redirect_uri=https%3A%2F%2Flink.springer.com%2Fnull%3Fidp_error%3Dcookies_not_supported&code=4fd6f2a7-d04b-407e-ae35-238137854ace HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: idp.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Content-Language: en-US
Expires: 0
Location: https://link.springer.com/null?idp_error=cookies_not_supported&error=cookies_not_supported&code=4fd6f2a7-d04b-407e-ae35-238137854ace
Pragma: no-cache
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-B3-Spanid: 496fe169d772bd97
X-B3-Traceid: 496fe169d772bd97
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Vcap-Request-Id: 283d7fd4-0c9c-40a7-6ceb-f8e24f8174e1
X-Xss-Protection: 1; mode=block
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:35:27 GMT
X-Served-By: cache-lhr7325-LHR
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1628184928.635209,VS0,VE18
Vary: x-forwarded-proto

---response end---
302 Found
Parsed Strict-Transport-Security max-age = 31536000, includeSubDomains = true
Updated HSTS host: idp.springer.com:443 (max-age: 31536000, includeSubdomains: true)
Location: https://link.springer.com/null?idp_error=cookies_not_supported&error=cookies_not_supported&code=4fd6f2a7-d04b-407e-ae35-238137854ace [following]
] done.
URI content encoding = None
--2021-08-05 18:35:27--  https://link.springer.com/null?idp_error=cookies_not_supported&error=cookies_not_supported&code=4fd6f2a7-d04b-407e-ae35-238137854ace
Found link.springer.com in host_name_addresses_map (0x55f6ab678630)
Connecting to link.springer.com (link.springer.com)|151.101.64.95|:443... connected.
Created socket 4.
Releasing 0x000055f6ab678630 (new refcount 1).

---request begin---
GET /null?idp_error=cookies_not_supported&error=cookies_not_supported&code=4fd6f2a7-d04b-407e-ae35-238137854ace HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: link.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: max-age=0
Expires: Thu, 05 Aug 2021 17:35:27 GMT
Location: /null?idp_error=cookies_not_supported
Server: Oscar Platform 0.314.0
X-Environment: live
X-Frame-Options: DENY
X-Origin-Server: 9d449640-d9d5-40e8-460a-733f
X-Vcap-Request-Id: d29da239-7637-4669-669f-7cdf6d53c4b9
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:35:27 GMT
Age: 0
X-Served-By: cache-f8e0b703-internal, cache-lon4270-LON
X-Cache: MISS, MISS
X-Cache-Hits: 0
X-Timer: S1628184928.734550,VS0,VE18

---response end---
302 Found
Disabling further reuse of socket 5.
Registered socket 4 for persistent reuse.
Location: /null?idp_error=cookies_not_supported [following]
] done.
URI content encoding = None
--2021-08-05 18:35:27--  https://link.springer.com/null?idp_error=cookies_not_supported
Reusing existing connection to link.springer.com:443.
Reusing fd 4.

---request begin---
GET /null?idp_error=cookies_not_supported HTTP/1.1
User-Agent: Wget/1.21.1
Accept: */*
Accept-Encoding: identity
Host: link.springer.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 302 Found
Connection: keep-alive
Content-Length: 0
Cache-Control: max-age=0
Expires: Thu, 05 Aug 2021 17:35:27 GMT
Location: https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie
Server: Oscar Platform 0.314.0
X-Environment: live
X-Frame-Options: DENY
X-Origin-Server: 1d728629-510b-47e9-4424-6d56
X-Vcap-Request-Id: c73b4957-d2d9-46ef-72a4-3444774f14f9
Via: 1.1 google, 1.1 varnish
X-Cdn-Origin: SNPaaS
Accept-Ranges: bytes
Date: Thu, 05 Aug 2021 17:35:27 GMT
Age: 0
X-Served-By: cache-bc5ba314-internal, cache-lon4270-LON
X-Cache: MISS, MISS
X-Cache-Hits: 0
X-Timer: S1628184928.764577,VS0,VE30

---response end---
302 Found
Location: https://idp.springer.com/authorize?redirect_uri=https://link.springer.com/null?idp_error=cookies_not_supported&client_id=springerlink&response_type=cookie [following]
] done.
URI content encoding = None
20 redirections exceeded.
Saving HSTS entries to /home/asinha/.wget-hsts

[geraldb]

Thanks for the HTTP headers that makes it clear - the second request (authorize) sets the cookies and the third request than needs to pass along the cookies to succeed - so you are right it's all about cookies. I try to put together a test script next week and if it works add the cookie support to the fetcher library /gem (that will update pluto).

Set-Cookie: idp_session=sVERSION_1b8226973-81df-4261-b453-7b221ddf56fd; Domain=.springer.com; Path=/; Secure; HttpOnly
Set-Cookie: idp_session_http=hVERSION_1598a3ad7-1ef8-49bf-9dd1-084eb935b38d; Domain=.springer.com; Path=/; HttpOnly
Set-Cookie: idp_marker=9e2e1951-e2ad-4669-8419-f747eefded20; Domain=.springer.com; Path=/; Max-Age=315360000; HttpOnl

[pacbard]

For what's worth, this script seems to work. The response.body variable returns the feed content for the springer journals that were redirecting before.

[geraldb]

Thanks for highlight the script on stackoverflow - I also searched / browsed some - one gotcha was:

cookie = response['Set-Cookie']

will only work if there's only one cookie but in the springer (or more generic case) there can be 3 or multiple. I try to get to it early next week. Cheers.