ScyllaDB insertion

Highlighted
Five Stars

ScyllaDB insertion

Has anyone successfully performed insertion with Open Studio for Big Data (7.1.1) into ScyllaDB?

 

I've inserted into single node [regular] Cassandra and multi- [3] node Cassandra without any problems... but can't get ScyllaDB Docker instance to insert at all.  Talend connects to the instance and creates keyspace and column family just fine if it does not exist... but does not insert data.  I can of course connect just fine to Scylla using cqlsh and any DB IDE.  ScyllaDB docker is running as localhost (127.0.0.1); has anyone had any luck with this?

Five Stars

Re: ScyllaDB insertion

...and of course, almost immediately after posting this (and clued in by a potential bit from the scylla.yaml documentation below) you need to uncheck the "Use unlogged batch" radio button from the tCassandraOutput component in order to get the insertions to transpire.

I'm still not sure why my "periodic" synch did not show the insertions as they should have shown up in short order; nevertheless, this does appear to be the solution for the interim period until I can figure out how to tweak this paramater more thoroughly. The snippet from my scylla.yaml follows:

Thanks,
Jonathan

# When in batch mode, Scylla won't ack writes until the commit log
# has been fsynced to disk. It will wait
# commitlog_sync_batch_window_in_ms milliseconds between fsyncs.
# This window should be kept short because the writer threads will
# be unable to do extra work while waiting. (You may need to increase
# concurrent_writes for the same reason.)
#
# commitlog_sync: batch
# commitlog_sync_batch_window_in_ms: 2
#
# the other option is "periodic" where writes may be acked immediately
# and the CommitLog is simply synced every commitlog_sync_period_in_ms
# milliseconds.
commitlog_sync: periodic
commitlog_sync_period_in_ms: 10000
Moderator

Re: ScyllaDB insertion

Hello,

So far, ScyllaDB is not list on the supported DB table.

For more information, please have a look at this online installation guide about:TalendHelpCenterSmiley Frustratedupported systems, databases and business applications by Talend components for talend open studio bigdata V 7.1.1

Feel free to create a workitem/new feature jira issue on talend bug tracker.

Best regards

Sabr

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog