sampling with TOS?

One Star

sampling with TOS?

Hi!
Are there any other methods to create a sample apart from tSampleRow?
I want a random sample or better a stratified sample.
if there is only tSampleRow please give me the reply that it isn´t possible to do it on an easy way.
thanks and best regards
matte

Community Manager

Re:sampling with TOS?

Hello

I want a random sample or better a stratified sample.


Would you like to get some randow rows? Can you give us an example?
Best regards
shong

----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re:sampling with TOS?

for example i have 1.000.000 Customers with different attributes and to optimize the processing time i want to get only 30 % of the whole Data. i dont want the first 300.000 customers. i want any of the customers (random).
and with stratified sample mean for example:
i have 1.000.000 customers too but i want a random sample that contains 40 % men and 60 % women.
like this:
Database:
id Name Gender ...
1 a m
2 b m
3 c f
4 d m
5 e f
6 f f
7 g f
8 h m
9 i m
10 j f
11 k m
12 l m
13 m f
14 n m
15 o f
16 p f
17 q f
18 r m
19 s m
20 t f

Sample 50 % with 40 % men and 60 %
id Name Gender ...
2 b m
3 c f
7 g f
8 h m
10 j f
12 l m
15 o f
17 q f
18 r m
20 t f
i hope this is more understanable

best regards
matte

One Star

Re:sampling with TOS?

i think there dont't have a componet can do this, i think you can use first get the ids of m and f
like
M's ID in DB
id
1
2
4
8
9
11
12
14
18
19


F's ID in DB
id
3
5
6
7
10
13
15
16
17
20
and difine a java method, use it to get random ids

Community Manager

Re:sampling with TOS?

Hello matte
As ylp_1 said, it need to define a Java method in a routine to get the random ids as your required, and then call this routine and get some random ids before quering data from table. Are you a Java programmer? If not, I can create a demo for you later.
Best regards
shong

----------------------------------------------------------
Talend | Data Agility for Modern Business
Six Stars

Re:sampling with TOS?

Hi!
Are there any other methods to create a sample apart from tSampleRow?
I want a random sample or better a stratified sample.
if there is only tSampleRow please give me the reply that it isn´t possible to do it on an easy way.
thanks and best regards
matte


If you are using a sql db as source, you can specify your select as
select * from mytable where rand() < 0.3
to fetch an approximate sample 30% of rows

One Star

Re:sampling with TOS?

thanks for your answers.
yes i´m a java programer. not the best but i can implement a random sampling.
i asked because i want to evaluate the function from tos that are given by the components.