One Star

tGoogleAnalyticsInput help required.

Hi,
I want to fetch Google Analytics (GA) data. For this, my friend added my google email as a user to his website GA. Now i can view his GA reports through my id. So this part is done.
I want to discuss three fields of tGoogleAnalyticsInput,
1.Application Name: Although it is not required but what should i enter here?
2.Service Account Email: i have given my google email address here as it has already been added as mentioned above.
3.Key File (*.p12): This is what confuses me. I've generated a p12 key file by turning my Analytics API to ON and following the required process.
My question is related to the 3rd field. Should i use p12 file that i generated or should i ask my friend to provide me his p12 key file?
any idea? Thanks in advance.
regards,
abdul.
42 REPLIES
One Star

Re: tGoogleAnalyticsInput help required.

Hi,
Attached is my component settings.
regards,
Seventeen Stars

Re: tGoogleAnalyticsInput help required.

At first, there is a documentation and there is always the possibility to contact me. There is absolute no reason to rate a component badly because of the lack of own knowlegde!
You should NOT use your own email address! You have to use the email address of your service account.
And to avoid your next stupid bad rating: Please add your service account email as user to all profiles you need to gather data.
Please keep in mind, the rating system should give users a hint about the quality of the component and not about the quality of developers which have less interest in communication and reading manuals.
One Star

Re: tGoogleAnalyticsInput help required.

Hi jlolling,
To be honest, i was looking for you on this post.
Sorry about that rating but my purpose, to post this link there, was just to gather attention of concerned persons. I will update my rating. Its is certainly worth appreciating if someone is working on custom components to make our tasks easier.
About the email address, I have added the service account email (see attached images) now but it still returns nothing.
btw, i tried fetching data from https://developers.google.com/apis-explorer/#s/analytics/v3/analytics.data.ga.get and it returns data correctly.
can u please explain a bit?
Thanks for replying.
regards,
abdul.
One Star

Re: tGoogleAnalyticsInput help required.

no need to be rude jlolling. As i said, i will update it Smiley Happy
Seventeen Stars

Re: tGoogleAnalyticsInput help required.

Sorry for my writing but it makes me angry if developers actually say: "your work is crap" because of a lack of knowledge. It is not the first time I am facing that problem.
Ok back to your problem: There is not result? That sound unusually. In your dimension is actually a metric, I am a bit surprised about this works for you.
I have tried your dimensions and metrics and get an exception because of the invalid usage of a metric as dimension.
You are sure there is no exception in your run console window?
Please take a look to the reference:
https://developers.google.com/analytics/devguides/reporting/core/v3/reference
Here you get an complete descriptions about dimensions and metrics, and please do not mix them up.
One Star

Re: tGoogleAnalyticsInput help required.

its perfectly ok.
well according to this https://developers.google.com/analytics/devguides/reporting/core/dimsmets/visitor#ga:visitCount
visitCount is a dimension. since it is optional, that is not my concern. There was no error with this dimension/metric combination as can be seen in attached image.
Since i'm taking my friend's help to add my email as a user to his webiste's GA, i've confusions about the whole process. Let me state steps i did to make it work.
1. My friend has a website with GA integrated, he added my google email as user. Now i was able to view hi stats through my google analytics home page.
2. Then i turned ON my Analytics API services and also added one service account. I also downloaded p12 key file associated to this service account.
3. My friend also added this service account email as user to his GA. He also provided me Profile Id.
That is all i need to make a data fetch request through tGoogleAnalyticsInput. But it returns nothing.
Please let me know that what is the point that i may be missing.
regards,
abdul
Seventeen Stars

Re: tGoogleAnalyticsInput help required.

Could you please check this:
Please select the google component and switch to the code view.
Between the section start and stop
must be a lot of code where you find your given attributes.
There is an annoying bug in Talend that prevents the job from getting the necessary code from the component.
Seventeen Stars

Re: tGoogleAnalyticsInput help required.

In which way you have installed the user components?
One Star

Re: tGoogleAnalyticsInput help required.

Thanks for looking into it in detail.
This is the code you asked for
"
/**
* start
*/
ok_Hash.put("tGoogleAnalyticsInput_1", false);
start_Hash.put("tGoogleAnalyticsInput_1",
System.currentTimeMillis());
currentComponent = "tGoogleAnalyticsInput_1";
int tos_count_tGoogleAnalyticsInput_1 = 0;
/**
* stop
*/
/**
* start
*/
"
and there is nothing in it.
I used this https://help.talend.com/search/all?query=Installing+a+custom+component&content-lang=en to install the custom component.
regards,
abdul.
Community Manager

Re: tGoogleAnalyticsInput help required.

There is an annoying bug in Talend that prevents the job from getting the necessary code from the component.

Hi Jan,
Sorry for popping up in your thread, but I suppose you have reported this issue already in JIRA?
Cheers,
Elisa
Seventeen Stars

Re: tGoogleAnalyticsInput help required.

Yes I have created an JIRA ticket to this issue.
@abdul: Please try this:
1. remove the file <studio_install_dir>/configuration/ComponentCache.javacache
2. check if the component is in the folder for user components (sometimes also called custom components)
3. open your preferences and navigate to Talend->Component and click on Apply.
Alternatively you can install the component via the Exchange view, this is by far the best way.
One Star

Re: tGoogleAnalyticsInput help required.

Hi
i want to keep this value in a column with the data i extracted from GA. any idea how can i put this return value(CONTAINS_SAMPLED_DATA) in the data for each row.
if not possible top put this in the data flow. i want to throw and exception in tjava if this value is true? and don't know how to acess this value ???
CONTAINS_SAMPLED_DATA
--------------------------------
want to acess this values and store them.
Return values
Return value
Content
ERROR_MESSAGE
Last error message
NB_LINE
Number of delivered lines
CONTAINS_SAMPLED_DATA
True if data are sampled, means not exactly
calculated. This can happen if you query to
many details.
SAMPLE_SIZE
The amount of datasets used for the query
SAMPLE_SPACE
The amount of available datasets for this query
TOTAL_AFFECTED_ROWS
Number of rows which are collected by Google
to calculate the result.
One Star

Re: tGoogleAnalyticsInput help required.

Hi jlolling
I need help with the tGoogleAnalyticsInput talend component. I have configured the job as per above comments but still im getting the below error. Please can you provide some help. Also attaching the screenshot of configuration and error.

Exception in component tGoogleAnalyticsInput_1
com.google.api.client.auth.oauth2.TokenResponseException: 400 Bad Request
{
  "error" : "invalid_grant"
}
Also i set the clock on my machine(On which talend is running) to sync time with internet. All screenshot attached below.

Seventeen Stars

Re: tGoogleAnalyticsInput help required.

Did you have enabled the access to the Analytics API in the Developer Console?
No this is not the problem. Please check if you have used the correct view-ID or better please check your request at first in the Google API Explorer.
Here the the google advices about errors:
https://developers.google.com/analytics/devguides/reporting/core/v3/coreErrors
and here the API explorer:
https://developers.google.com/apis-explorer/#s/analytics/v3/analytics.data.ga.get
One Star

Re: tGoogleAnalyticsInput help required.

Thanks jlolling for the response,
All of the configuration were valid.but the issue was related to my silly mistake, there was leading space in the service account email ID in entered.It is resolved and job worked perfectly.
Thanks
Puneet
Seventeen Stars

Re: tGoogleAnalyticsInput help required.

Ok, I will take care in the next release such kind of mistakes does not matter anymore. 
One Star

Re: tGoogleAnalyticsInput help required.

I get a similar error:
Exception in component tGoogleAnalyticsInput_1
com.google.api.client.auth.oauth2.TokenResponseException: 400 Bad Request
{
  "error" : "invalid_grant"
}
    at com.google.api.client.auth.oauth2.TokenResponseException.from(TokenResponseException.java:105)
    at com.google.api.client.auth.oauth2.TokenRequest.executeUnparsed(TokenRequest.java:287)
    at com.google.api.client.auth.oauth2.TokenRequest.execute(TokenRequest.java:307)
    at com.google.api.client.googleapis.auth.oauth2.GoogleCredential.executeRefreshToken(GoogleCredential.java:384)
    at com.google.api.client.auth.oauth2.Credential.refreshToken(Credential.java:489)
    at com.google.api.client.auth.oauth2.Credential.intercept(Credential.java:217)
    at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:859)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:410)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:343)
disconnected
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:460)
    at de.jlo.talendcomp.googleanalytics.GoogleAnalyticsInput.executeDataQuery(GoogleAnalyticsInput.java:352)
    at de.jlo.talendcomp.googleanalytics.GoogleAnalyticsInput.executeQuery(GoogleAnalyticsInput.java:417)
    at test.googleanalytics_0_1.googleAnalytics.tGoogleAnalyticsInput_1Process(googleAnalytics.java:575)
    at test.googleanalytics_0_1.googleAnalytics.runJobInTOS(googleAnalytics.java:1058)
    at test.googleanalytics_0_1.googleAnalytics.main(googleAnalytics.java:916)
No clues...
Seventeen Stars
One Star

Re: tGoogleAnalyticsInput help required.

jlolling,
Great piece of code. Thanks.
Quick question what is the best way to get the meta data from the various Google Analytics reports? I saw that someone was using a PreJob component before the tGoogleAnalyticsInput module. Is that required?
Thank sin advance.

Patrick
Seventeen Stars

Re: tGoogleAnalyticsInput help required.

It depends what you mean with meta data. I usually use another framework (job instance framework based on the components:
https://exchange.talend.com/#marketplaceproductoverview:gallery=marketplace%252F1&pi=marketplace%252...
 to register all jobs (not only the Google Analytics stuff) to track to procedure and steer the jobs, it is used for the same use case as the AMC but does the job more related to DWH needs.
If you mean meta data from Google Analytics it self, than you could use tGoogleAnalyticsManagement component to get the most of it. 
One Star

Re: tGoogleAnalyticsInput help required.

jlolling,
That helped.  But now I am getting a 403 error.
WARN:Got error:403 Forbidden
{
  "code" : 403,
  "errors" : ,
  "message" : "User does not have sufficient permissions for this profile."
}
Seventeen Stars

Re: tGoogleAnalyticsInput help required.

Simply add the service account email address to the list of the users for the related views (formally known as profiles).
One Star

Re: tGoogleAnalyticsInput help required.

jlolling,
That worked.
Thanks.
One Star

Re: tGoogleAnalyticsInput help required.

jlolling,
I am following your example "Scenario 1 - Using flat and normalized output in a test job".  I am getting two errors:
1) The method convertToDatatype (String, String, String, String)
2)  Syntax error on token "en",, expected
Thanks in advance.
Seventeen Stars

Re: tGoogleAnalyticsInput help required.

I cannot understand how this can happened because the only java code which can fail this way is the code in the talend job.
You shows me a method inside the library and actually this cannot fail because it is already complied.
Could you please send me a screenshot of the job and the configuration please. It would also interesting which other Google related components you run in the same job (also in parent jobs). 
I can tell you it must be a configuration problem because this component works in huge projects without any problems for a long time.
If you have problems with the forum here (sometimes people are unable to send screenshots) you can contact me directly.
jan.lolling@gmail.com
I am always interested in trouble situations like this to improve my work or have ideas to avoid such situations.
: Did you have added quotas around the locale like this way: "en". Sorry the component expect here the en without quotas.
I should take care of it and remove them automatically if there are such quotas. 
One Star

Re: tGoogleAnalyticsInput help required.

Simply add the service account email address to the list of the users for the related views (formally known as profiles).

hi, 
i have the same problem but i added the service account in user management and the problem exist yet.
Any idea?
thanks, Cynthia
Seventeen Stars

Re: tGoogleAnalyticsInput help required.

Hi piruvariCynthia
This should work. What kind of problems do you get?
You also have to enable the Analytics API for this service account.
One Star

Re: tGoogleAnalyticsInput help required.

Hi Jan,
I need your help, I am getting a Null Pointer Exception in the method GoogleAnalyticsInput.containsSampledData. 
I tried the same query on google api explorer and I get proper result. Attaching few snapshots - any help,support,pointers, etc.. will be greatly appreciated.
 
Regards,
Boyai
One Star

Re: tGoogleAnalyticsInput help required.

Opps... the snapshots has gone missing.

Starting job GoogleAnalytics at 11:27 08/09/2016.
connecting to socket on port 3494
connected
DEBUG:add dimension: {name=ga:visitCount}
DEBUG:add metric: {expression=ga:sessions}
Exception in component tGoogleAnalyticsInput_1
java.lang.NullPointerException
at de.jlo.talendcomp.google.analytics.v4.GoogleAnalyticsInput.containsSampledData(GoogleAnalyticsInput.java:655)
at local_project.googleanalytics_0_1.GoogleAnalytics.tGoogleAnalyticsInput_1Process(GoogleAnalytics.java:578)
at local_project.googleanalytics_0_1.GoogleAnalytics.runJobInTOS(GoogleAnalytics.java:864)
at local_project.googleanalytics_0_1.GoogleAnalytics.main(GoogleAnalytics.java:721)
disconnected
Job GoogleAnalytics ended at 11:27 08/09/2016.

API Explorer
POST {YOUR_API_KEY}
{
"reportRequests": ,
"viewId": "ga:871246",
"metrics": ,
"dimensions":
}
]
}