TOS 5.X tFileFetch dowloading CSV from Redmine System HTTPS problem

One Star

TOS 5.X tFileFetch dowloading CSV from Redmine System HTTPS problem

Mission:
Download with Talend Components tFileFetch CSV Reports from a Redmine System
Redmine needs Login, is configured to use HTTPS and has redirects
tried Implementation#1:
tFileFetch_1
using POST, redirect=true, PostVars with LoginCredentials, save Cookie, single Cookie true, Url: Login Page
|
|
onSubjobOK Link
|
|
tFileFetch_2
using Get, readCookie, URL: Redmine Path to the CSV data
ISSUE #1: There isContinue_tFileFetch_1 an exception on: https://XXX/redmine/login
java.lang.Exception: Method failed: HTTP/1.1 500 Internal Server Error
tried Implementation#2:
same as #1 without onSubjobOK Link
#ISSUE #2: tFileFetch_2 causes an exception "Method failed: HTTP/1.1 406 Not Acceptable"

QUESTIONS:
How to implement this job scenario?

WIN 7 - Version: 5.0.2 Build id: r78327-20120216-0244 - JAVA
One Star

Re: TOS 5.X tFileFetch dowloading CSV from Redmine System HTTPS problem

Hi
Just use only one tFileFetch.
Need Authentication: true.
Support Redirection: true.
Besides, the URI shouldn't be Login Page. It should be like this "http://talendforge.org/file_fetch.txt".
Regards,
Pedro
One Star

Re: TOS 5.X tFileFetch dowloading CSV from Redmine System HTTPS problem

Hi Pedro,
thanks for your reply.
I already did this in my first tries.
tFileFetch:
Authentication (unnecessary as it is HTTP Authentication but anyway ticked on): Login Credentials
Use Post: provided Logincredentials + plus redirection url info to Url for downloading the csv
Write / read cookie (write for login, read for the redirect)
ticked on redirect
I did also test a lot of scenarios e.g. with the url:
- direct CSV download URL
- login URL and redirect url
- etc
But this implementation did not work. I encountered the issue, that the session cookie was not set properly and the redirect failed. Therefore I started the test for an implementation similiar as mentioned in the Component Guide PDF, provided by Talend.
Status: Issue is still unresolved
One Star

Re: TOS 5.X tFileFetch dowloading CSV from Redmine System HTTPS problem

Hi
Could you tell me some info to reproduce this issue?
Or I can figure out how to fix it.
Regards,
Pedro
One Star

Re: TOS 5.X tFileFetch dowloading CSV from Redmine System HTTPS problem

Hi Pedro
I did following tests:
Using one tFileFetch and configured as following:
https
PostMethod
PostVariables: Logincredentials as HTML Formfieldnames from the LoginPage
Save Cookie: true (creation by Login)
Read Cookie: true (reading while redirecting to requested page)
Single Cookie header: true
Response to Console: true
Support Redirect: true <--- check Exception Test 2
Additional: Authentication on, Login Credentials filled out
Test1: URL to get the CSV of Tickets for a particular Tracker:
URL: https://XXX/redmine/projects/ABC/issues.csv?c%5B%5D=project&c%5B%5D=tracker&c%5B%5D=status&c%5B%5D=priority&c%5B%5D=subject&c%5B%5D=author&c%5B%5D=assigned_to&c%5B%5D=updated_on&c%5B%5D=category&c%5B%5D=cf_16&f%5B%5D=status_id&f%5B%5D=tracker_id&f%5B%5D=&group_by=&op%5Bstatus_id%5D=%2A&op%5Btracker_id%5D=%3D&set_filter=1&v%5Bstatus_id%5D%5B%5D=1&v%5Btracker_id%5D%5B%5D=18

Result: Exception:
There isContinue_tFileFetch_1 an exception on: https://#####.....
java.lang.Exception: Method failed: HTTP/1.1 406 Not Acceptable
at jobexport.loadredmine_0_1.LoadRedmine.tFileFetch_1Process(LoadRedmine.java:517)
at jobexport.loadredmine_0_1.LoadRedmine.runJobInTOS(LoadRedmine.java:834)
at jobexport.loadredmine_0_1.LoadRedmine.main(LoadRedmine.java:691)

Test 2: Url to Webpage with the Tickets and try to save the Webpage
URL: https://XXX/redmine/projects/YYY/issues?set_filter=1
Result: Exception:
org.apache.commons.httpclient.HttpMethodDirector isRedirectNeeded
INFO: Redirect requested but followRedirects is disabled
Redirect to: https://XXX/redmine/login?back_url=https%3A%2F%2FXXX%2Fredmine%2Fprojects%2FYYY%2Fissues
Status Line: HTTP/1.1 200 OK
*** Response Header ***
Date: Wed, 09 May 2012 12:15:02 GMT
Server: Mongrel 1.1.5
Status: 200
ETag: "53d25a9c8844ce24f82e333c2b1c45af"
X-Runtime: 12
Content-Type: text/html; charset=utf-8
Content-Length: 3794
Cache-Control: private, max-age=0, must-revalidate
Set-Cookie: _redmine_session=BAh7CDoQX2NzcmZfdG9rZW4iMTFCNDRsM1lsUS9lMVlZblFRcmpickFOWWpXN3NVVjI5dkYwRWs4aXdZUkU9Og9zZXNzaW9uX2lkIiU4Y2U2MWNhNmNlYjA2ZWM3NzJjNzcwMjczOTk0NWRhZCIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNoSGFzaHsGOgplcnJvciIqQmVudXR6ZXIgb2RlciBLZW5ud29ydCBpc3QgdW5nw7xsdGlnLgY6CkB1c2VkewY7CFQ%3D--30b10957e5df0449265ba58a6b25203bbf8f0173; path=/; HttpOnly
I got a 200 OK Response for a LoginPage on which I was redirected as I have to log in first. (Kindly note the url addition back_url)
Thanks For Support

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Agile Data lakes & Analytics

Accelerate your data lake projects with an agile approach

Watch