tMongoDBInput Aggregation Pipeline query

Four Stars

tMongoDBInput Aggregation Pipeline query

Hello All,

 

I'm trying to to put together a Aggregation Pipeline query in tMongoDBInput and it doesn't work(I get no records out) irrespective of whatever condition I use for filtering by date. If I remove the condition for date, everything works fine.

 

Attached a screenshot of what condition I have and the context value is 2017-06-24 08:07:28.113Z.

 

It would be great if somebody can provide some assistance.

Four Stars

Re: tMongoDBInput Aggregation Pipeline query

Is there not a solution for this?

Moderator

Re: tMongoDBInput Aggregation Pipeline query

Hi,

Sorry for late response.

Can you successfully execute your filter by using 2017-06-24 08:07:28.113Z without using context value?

On which build version you got this issue?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Four Stars

Re: tMongoDBInput Aggregation Pipeline query

Hello Sabrina,

 

No problem. I use TOS Big Data 6.4.1.

This one - {$match: {order_date:{$gt: new Date("2017-06-24 08:07:28.113Z")}}}, works directly in the Mongo shell, but when I use it in MongodbInput as "{$match: {order_date:{$gt: new Date(\"2017-06-24 08:07:28.113'Z'\")}}}", I get the following error after the build,

 

Exception in component tMongoDBInput_1 (ItemInSalesOrder_Copy_Copy)
com.mongodb.util.JSONParseException:
{$match: {order_date:{$gt: new Date("2017-06-24 08:07:28.113'Z'")}}}
^
at com.mongodb.util.JSONParser.read(JSON.java:280)
at com.mongodb.util.JSONParser.parse(JSON.java:171)
at com.mongodb.util.JSONParser.parseObject(JSON.java:241)
at com.mongodb.util.JSONParser.parse(JSON.java:205)
at com.mongodb.util.JSONParser.parseObject(JSON.java:241)
at com.mongodb.util.JSONParser.parse(JSON.java:205)
at com.mongodb.util.JSONParser.parseObject(JSON.java:241)
at com.mongodb.util.JSONParser.parse(JSON.java:205)
at com.mongodb.util.JSONParser.parse(JSON.java:155)
at com.mongodb.util.JSON.parse(JSON.java:93)
at com.mongodb.util.JSON.parse(JSON.java:74)
at bd_newproject.iteminsalesorder_copy_copy_0_1.ItemInSalesOrder_Copy_Copy.tMongoDBInput_1Process(ItemInSalesOrder_Copy_Copy.java:2234)
at bd_newproject.iteminsalesorder_copy_copy_0_1.ItemInSalesOrder_Copy_Copy.runJobInTOS(ItemInSalesOrder_Copy_Copy.java:4619)
at bd_newproject.iteminsalesorder_copy_copy_0_1.ItemInSalesOrder_Copy_Copy.main(ItemInSalesOrder_Copy_Copy.java:4436)

 

When I change it this way - "{$match: {order_date:{$gt: '" + TalendDate.formatDate("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'", "2017-06-24 08:07:28.113'Z'") + "'}}}",

 

I get the error(attached), when I press run(before the build completes).

 

Four Stars

Re: tMongoDBInput Aggregation Pipeline query

Hello Sabrina,

 

No solution yet?

Moderator

Re: tMongoDBInput Aggregation Pipeline query

Hello,

We are making an investigation on your issue and then come back to you as soon as we can.

Thank for your time.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Four Stars

Re: tMongoDBInput Aggregation Pipeline query

Thank you Sabrina. Will wait for your findings.

 

By the way, the datatype for the field in DB is Date.

Five Stars

Re: tMongoDBInput Aggregation Pipeline query

I have the same problem
there any solution ?
Moderator

Re: tMongoDBInput Aggregation Pipeline query

Hello,

Could you please show us your query in tmongodbinnput?

Here are related success use cases and feel free to let us know if it helps for you.

https://community.talend.com/t5/Design-and-Development/tMongoDBInput-Exception-when-using-match-stag...

https://community.talend.com/t5/Design-and-Development/aggregation-query-in-tmongoDBInput/m-p/54857

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

Calling Talend Open Studio Users

The first 100 community members completing the Open Studio survey win a $10 gift voucher.

Start the survey

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now