One Star

Need to get previous date using getcurrentdate() in tftpget component

Folks,
I'm a talend newbie and a java newbie as well. I need some help from the experts here. I have a scenario where in my files are in remote location saying "remotefiledir/2013/05/01/clientid/xml1".
Similarly I have files in 2013/04/30/clientid/xml2. Now I need to get the previous day files using tftpget component. The current code would get the files 2013/05/01 when the current_date is 2013/05/02. But when the current date is 2013/05/01 it is not getting the file in location 2013/04/30 instead trying to get in 2013/05/00
Current code :
TalendDate.formatDate("yyyy",TalendDate.getCurrentDate())
+ "/" +TalendDate.formatDate("MM",TalendDate.getCurrentDate() )
+ "/" +TalendDate.formatDate("dd",TalendDate.addDate(TalendDate.getCurrentDate(),-1,"dd")
What is the way to use any talend routines in tftpget component which can get me the previous day files? If this can be achieved in tjavarow component..I need the sample code as well...Appreciate your help in this!!
5 REPLIES
Seventeen Stars

Re: Need to get previous date using getcurrentdate() in tftpget component

Of course, you should create your own routine to do that. A routine is quit easy to create. Simply create a class under the Code node in Studio and create a static method. The way to document it is a bit strange but that helps to have code completion in your job for your routine.
Here an example of what you are doing with the build routines:
Here a Routine class which I have written for a couple of timestamp and date manipulation. You would use the addDays method (like you with -1 to get yesterday).
To create a routine simple click right on the routines node and create a routine called TimestampUtil. After that copy&past this content in the routine (replace all existing), that it. Happy coding.
package routines;
import java.text.SimpleDateFormat;
import java.util.Calendar;
import java.util.Locale;
/*
* user specification: the function's comment should contain keys as follows: 1. write about the function's comment.but
* it must be before the "{talendTypes}" key.
*
* 2. {talendTypes} 's value must be talend Type, it is required . its value should be one of: String, char | Character,
* long | Long, int | Integer, boolean | Boolean, byte | Byte, Date, double | Double, float | Float, Object, short |
* Short
*
* 3. {Category} define a category for the Function. it is required. its value is user-defined .
*
* 4. {param} 's format is: {param} <type> <name>
*
* <type> 's value should be one of: string, int, list, double, object, boolean, long, char, date. <name>'s value is the
* Function's parameter name. the {param} is optional. so if you the Function without the parameters. the {param} don't
* added. you can have many parameters for the Function.
*
* 5. {example} gives a example for the Function. it is optional.
*/
public class TimestampUtil {

/**
* getTimeframeStart: returns the date at the start of a timeframe.
*
* {Category} TimestampUtil
*
* {talendTypes} Date
*
* {param} Date() timestamp: The timestamp within a timeframe.
*
* {example} getTimeframeStart(TalendDate.getCurentDate()).
*/
public static java.util.Date getTimeframeStart(java.util.Date timestamp) {
java.util.Calendar c = java.util.Calendar.getInstance();
c.setTime(timestamp);
// cut minutes and seconds
c.set(java.util.Calendar.MINUTE, 0);
c.set(java.util.Calendar.SECOND, 0);
c.set(java.util.Calendar.MILLISECOND, 0);
return c.getTime();
}

/**
* getTimeframeEnd: returns the date at the end of a timeframe.
*
* {Category} TimestampUtil
*
* {talendTypes} Date
*
* {param} Date() timestamp: The timestamp within a timeframe.
*
* {example} getTimeframeEnd(TalendDate.getCurentDate()).
*/
public static java.util.Date getTimeframeEnd(java.util.Date timestamp) {
java.util.Calendar c = java.util.Calendar.getInstance();
c.setTime(timestamp);
// cut minutes and seconds
c.set(java.util.Calendar.MINUTE, 0);
c.set(java.util.Calendar.SECOND, 59);
c.set(java.util.Calendar.MILLISECOND, 999);
return c.getTime();
}
/**
* getNextDay: returns the start of the next day.
*
* {Category} TimestampUtil
*
* {talendTypes} Date
*
* {param} Date() day: any day.
*
* {example} getNextDay(TalendDate.getCurentDate()).
*/
public static java.util.Date getNextDay(java.util.Date day) {
java.util.Calendar c = java.util.Calendar.getInstance();
c.setTime(day);
// cut minutes and seconds
c.set(java.util.Calendar.MINUTE, 0);
c.set(java.util.Calendar.SECOND, 0);
c.set(java.util.Calendar.MILLISECOND, 0);
c.add(java.util.Calendar.HOUR, 24);
return c.getTime();
}

/**
* truncateToDay: returns the start of the given day.
*
* {Category} TimestampUtil
*
* {talendTypes} Date
*
* {param} Date() day: any day.
*
* {example} truncateToDay(TalendDate.getCurentDate()).
*/
public static java.util.Date truncateToDay(java.util.Date timestamp) {
if(timestamp == null) return null;
java.util.Calendar c = java.util.Calendar.getInstance();
c.setTime(timestamp);
// cut time
c.set(java.util.Calendar.MINUTE, 0);
c.set(java.util.Calendar.SECOND, 0);
c.set(java.util.Calendar.MILLISECOND, 0);
c.set(java.util.Calendar.HOUR_OF_DAY, 0);
return c.getTime();
}
/**
* truncateToDay: returns the start of the given day.
*
* {Category} TimestampUtil
*
* {talendTypes} Date
*
* {param} Date() day: any day.
*
* {param} int framelagnth: length of the frame in minutes. (if framelength is not an integer devider of 60, last frame will be shorter)
*
* {example} truncateToMinuteFrame(TalendDate.getCurentDate(),5).
*/
public static java.util.Date truncateToMinuteFrame(java.util.Date timestamp, int framelength) {
if(timestamp == null) return null;
java.util.Calendar c = java.util.Calendar.getInstance();
c.setTime(timestamp);
// cut time
c.set(java.util.Calendar.MINUTE, (int)(c.get(Calendar.MINUTE)/framelength)*framelength);
c.set(java.util.Calendar.SECOND, 0);
c.set(java.util.Calendar.MILLISECOND, 0);
return c.getTime();
}

/**
* truncateToMonth: returns the first day of the month of the given day.
*
* {Category} TimestampUtil
*
* {talendTypes} Date
*
* {param} Date() day: any day.
*
* {example} truncateToMonth(TalendDate.getCurentDate()).
*/
public static java.util.Date truncateToMonth(java.util.Date timestamp) {
if(timestamp == null) return null;
// cut time
timestamp = truncateToDay(timestamp);
java.util.Calendar c = java.util.Calendar.getInstance();
c.setTime(timestamp);
// cut day
c.set(java.util.Calendar.DAY_OF_MONTH, 1);
return c.getTime();
}
/**
* truncateToWeek: returns the first day of the week of the given day.
*
* {Category} TimestampUtil
*
* {talendTypes} Date
*
* {param} Date() day: any day.
*
* {example} truncateToWeek(TalendDate.getCurentDate()).
*/
public static java.util.Date truncateToWeek(java.util.Date timestamp, java.util.Locale locale) {
if(timestamp == null) return null;
// cut time
timestamp = truncateToDay(timestamp);
java.util.Calendar c = java.util.Calendar.getInstance(locale);
c.setTime(timestamp);
// cut day
c.set(java.util.Calendar.DAY_OF_WEEK, c.getFirstDayOfWeek());
return c.getTime();
}
/**
* truncateToWeek: returns the first day of the week of the given day with Germany as locale.
*
* {Category} TimestampUtil
*
* {talendTypes} Date
*
* {param} Date() day: any day.
*
* {example} truncateToWeek(TalendDate.getCurentDate()).
*/
public static java.util.Date truncateToWeek(java.util.Date timestamp) {
return truncateToWeek(timestamp, java.util.Locale.GERMANY);
}
/**
* getUTC: returns the UTC time.
* @param siteTimestamp Timestamp from source data
* @return Date with UTC time
*
* {talendTypes} Date
*
* {Category} TimestampUtil
*
* {param} date() siteTimestamp: any day.
*
* {example} getUTC(TalendDate.getCurentDate()).
*/
public static java.util.Date getUTC(java.util.Date siteTimestamp) {
return getUTC(java.util.Locale.GERMANY, siteTimestamp);
}
/**
* getUTC: returns the UTC time.
* @param local country representing the time zone from source data
* @param siteTimestamp Timestamp from source data
* @return Date with UTC time
*
* {talendTypes} Date
*
* {Category} TimestampUtil
*
* {param} Locale(locale) locale: from which country
*
* {param} date(siteTimestamp) siteTimestamp : the date value
*
* {example} getUTC(Locale.SPAIN, TalendDate.getCurentDate()).
*/
public static java.util.Date getUTC(java.util.Locale locale, java.util.Date siteTimestamp) {
if (siteTimestamp != null) {
java.util.Calendar c = java.util.Calendar.getInstance(locale);
c.setTimeInMillis(siteTimestamp.getTime());
java.util.TimeZone z = c.getTimeZone();
int offset = z.getOffset(siteTimestamp.getTime());
int offsetHrs = offset / 1000 / 60 / 60;
int offsetMins = offset / 1000 / 60 % 60;
c.add(java.util.Calendar.HOUR_OF_DAY, (-offsetHrs));
c.add(java.util.Calendar.MINUTE, (-offsetMins));
return c.getTime();
} else {
return null;
}
}

/**
* addMinutes: returns modified date
* @param today any day
* @param MinutesToAdd Minutes to add
* @return new date_time
*
* {talendTypes} Date
*
* {Category} TimestampUtil
*
* {param} date today: any day.
*
* {param} long day: days to be added. (-1 for yesterday)
*
* {example} addDays(today, daysToAdd).
*/
public static java.util.Date addMinutes(java.util.Date today, long minutesToAdd) {
if (today != null) {
return new java.util.Date(today.getTime() + (1000 * 60 * minutesToAdd));
} else {
return null;
}
}
/**
* addDays: returns date of yesterday
* @param today any day
* @param daysToAdd difference in days
* @return Date with UTC time
*
* {talendTypes} Date
*
* {Category} TimestampUtil
*
* {param} date today: any day.
*
* {param} long day: days to be added. (-1 for yesterday)
*
* {example} addDays(today, daysToAdd).
*/
public static java.util.Date addDays(java.util.Date today, long daysToAdd) {
if (today != null) {
return new java.util.Date(today.getTime() + (1000 * 60 * 60 * 24 * daysToAdd));
} else {
return null;
}
}
/**
* addMonths: returns date + given months
* @param today any day
* @param monthsToAdd difference in months
* @return Date with UTC time
*
* {talendTypes} Date
*
* {Category} TimestampUtil
*
* {param} date today: any day.
*
* {param} int month: months to be added. (-1 for yesterday)
*
* {example} addDays(today, monthsToAdd).
*/
public static java.util.Date addMonths(java.util.Date today, int monthsToAdd) {
if (today != null) {
java.util.Calendar c = java.util.Calendar.getInstance();
c.setTime(today);
c.add(Calendar.MONTH, monthsToAdd);
return c.getTime();
} else {
return null;
}
}

/**
* returns the younger of both
* @param date1
* @param date2
* @return the younger of both
*
* {talendTypes} Date
*
* {Category} TimestampUtil
*
* {param} Date date1: any day.
*
* {param} Date date2: any day
*
* {example} getYounger(date1, date2).
*/
public static java.util.Date getYounger(java.util.Date date1, java.util.Date date2) {
if (date1 != null && date2 != null) {
if (date1.after(date2)) {
return date1;
} else {
return date2;
}
} else if (date1 != null) {
return date1;
} else if (date2 != null) {
return date2;
} else {
return null;
}
}
/**
* returns the older of both
* @param date1
* @param date2
* @return the older of both
*
* {talendTypes} Date
*
* {Category} TimestampUtil
*
* {param} Date date1: any day.
*
* {param} Date date2: any day
*
* {example} getOlder(date1, date2).
*/
public static java.util.Date getOlder(java.util.Date date1, java.util.Date date2) {
if (date1 != null && date2 != null) {
if (date1.before(date2)) {
return date1;
} else {
return date2;
}
} else if (date1 != null) {
return date1;
} else if (date2 != null) {
return date2;
} else {
return null;
}
}

/**
* getCurrentDayInyyyyMMdd: returns the current day.
*
* {talendTypes} String
*
* {Category} TimestampUtil
*
* {example} getCurrentDayInyyyyMMdd().
*/
public static String getCurrentDayInyyyyMMdd() {
return new SimpleDateFormat("yyyyMMdd").format(new java.util.Date());
}

/**
* getCurrentDayInyyyyMMdd: returns the current day.
*
* {talendTypes} String
*
* {Category} TimestampUtil
*
* {param} Date(date) date : date to format
*
* {example} getCurrentDayInyyyyMMdd().
*/
public static String formatInyyyyMMdd(java.util.Date date) {
if (date != null) {
return new SimpleDateFormat("yyyyMMdd").format(date);
} else {
return null;
}
}
/**
* format: returns the formatted date as String.
*
* {talendTypes} String
*
* {Category} TimestampUtil
*
* {param} Date(date) date : date to format
*
* {param} String(pattern) pattern : date to format
*
* {example} format(date, "dd.MM.yyyy")
*/
public static String format(java.util.Date date, String pattern) {
if (date != null) {
return new SimpleDateFormat(pattern).format(date);
} else {
return null;
}
}

/**
* getLocal: returns the local time.
*
* {talendTypes} Date
*
* {Category} TimestampUtil
*
* {param} string("Europe/Madrid") localTimeZoneId : ID of time zone
*
* {param} date(utcTimestamp) utcTimestamp : the date value
*
* {example} getLocalTimeFromUTC("Europe/Madrid", utcTimestamp).
*/
public static java.util.Date getLocalTimeFromUTC(String localTimeZoneId, java.util.Date utcTimestamp) {
java.util.TimeZone tz = getTimeZone(localTimeZoneId);
return getLocalTimeFromUTC(tz, utcTimestamp);
}
/**
* getLocal: returns the local time.
*
* {talendTypes} Date
*
* {Category} TimestampUtil
*
* {param} string("Europe/Madrid") localTimeZoneId : ID of time zone
*
* {param} date(mezTimestamp) utcTimestamp : the date value
*
* {example} getLocalTimeFromMEZ("Europe/Madrid", mezTimestamp).
*/
public static java.util.Date getLocalTimeFromMEZ(String localTimeZoneId, java.util.Date mezTimestamp) {
java.util.TimeZone tz = getTimeZone(localTimeZoneId);
return getLocalTimeFromMEZ(tz, mezTimestamp);
}
private static java.util.TimeZone getTimeZone(String idParam) {
if (idParam == null || idParam.isEmpty()) {
throw new IllegalArgumentException("id for time zone cannot be null or empty");
}
idParam = idParam.trim();
String[] tzArray = java.util.TimeZone.getAvailableIDs();
for (String id : tzArray) {
if (id.equalsIgnoreCase(idParam)) {
return java.util.TimeZone.getTimeZone(id);
}
}
return null;
}
private static java.util.TimeZone utcTimeZone = null;
private static java.util.TimeZone mezTimeZone = null;
private static java.util.TimeZone getUTCTimeZone() {
if (utcTimeZone == null) {
utcTimeZone = java.util.TimeZone.getTimeZone("UTC");
}
return utcTimeZone;
}

private static java.util.TimeZone getMEZTimeZone() {
if (mezTimeZone == null) {
mezTimeZone = java.util.TimeZone.getTimeZone("Europe/Berlin");
}
return mezTimeZone;
}
private static java.util.Date getLocalTimeFromUTC(java.util.TimeZone tz, java.util.Date utcTimestamp) {
return getLocalTime(tz, utcTimestamp, getUTCTimeZone());
}
private static java.util.Date getLocalTimeFromMEZ(java.util.TimeZone tz, java.util.Date mezTimestamp) {
return getLocalTime(tz, mezTimestamp, getMEZTimeZone());
}
private static java.util.Date getLocalTime(java.util.TimeZone localTimeZone, java.util.Date timestamp, java.util.TimeZone sourceTimeZone) {
if (timestamp != null) {
if (localTimeZone == null) {
throw new IllegalArgumentException("localTimeZone cannot be null");
}
if (sourceTimeZone == null) {
throw new IllegalArgumentException("sourceTimeZone cannot be null");
}
java.util.Calendar c = java.util.Calendar.getInstance();
//c.setTimeZone(sourceTimeZone);
c.clear();
c.setTimeInMillis(timestamp.getTime());
// adapt the local time zone
int offset = localTimeZone.getOffset(timestamp.getTime());
offset = offset - sourceTimeZone.getOffset(timestamp.getTime());
int offsetHrs = offset / 1000 / 60 / 60;
int offsetMins = offset / 1000 / 60 % 60;
c.add(java.util.Calendar.HOUR_OF_DAY, (offsetHrs));
c.add(java.util.Calendar.MINUTE, (offsetMins));
return c.getTime();
} else {
return null;
}
}
/**
* getTimeRatioInRange: returns relative position of a test date in a time range
*
* {talendTypes} double
*
* {Category} TimestampUtil
*
* {param} date(mesaure_date) test : test date
*
* {param} date(max_date) rangeEnd : the end of range
*
* {param} int(7) workDateRange : number of dates for range (regardless of sign)
*
* {example} getTimeRatioInRange(measure_date, max_date, workDateRange).
*/
public static double getTimeRatioInRange(java.util.Date test, java.util.Date rangeEnd, long workDateRange) {
if (test == null || rangeEnd == null) {
return 0;
}
long oneDayInMillis = 24l * 60l * 60l * 1000l;
workDateRange = Math.abs(workDateRange);
long workDateRangeInMillis = workDateRange * oneDayInMillis;
long testDateInMillis = test.getTime();
long rangeEndInMillis = rangeEnd.getTime();
long diff = rangeEndInMillis - testDateInMillis + oneDayInMillis;
if (diff > (oneDayInMillis)) {
return (double) workDateRangeInMillis / diff;
} else {
return workDateRange;
}
}

public static int getDaysBetween(java.util.Date beginDate, java.util.Date endDate ) {
long oneDayInMillis = 24 * 60 * 60 * 1000;
return (int)((endDate.getTime()-beginDate.getTime())/oneDayInMillis);
}

/**
* getSQLForWeekMonday: sql for sunday
* @param today any day
* @return Date with no time
*
* {talendTypes} String
*
* {Category} TimestampUtil
*
* {param} string week: German week in yyyy-ww.
*
* {example} getSQLForWeekMonday(week).
*/
public static String getSQLForWeekMonday(String weekString) throws java.text.ParseException {
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-ww",new Locale("de","DE")); // we use DIN 1355/ISO 8601 weeks
java.util.Date startDate = getLastMonday(sdf.parse(weekString));
SimpleDateFormat sdfOutput = new SimpleDateFormat("dd.MM.yyyy");
return "to_date('" + sdfOutput.format(startDate) + "','DD.MM.YYYY')";
}

/**
* getSQLForWeekSunday: sql for sunday
* @param today any day
* @return Date with no time
*
* {talendTypes} String
*
* {Category} TimestampUtil
*
* {param} string week: week in yyyy-ww.
*
* {example} getSQLForWeekSunday(week).
*/
public static String getSQLForWeekSunday(String weekString) throws java.text.ParseException {
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-ww",new Locale("de","DE"));// we use DIN 1355/ISO 8601 weeks
java.util.Date startDate = getNextSunday(sdf.parse(weekString));
SimpleDateFormat sdfOutput = new SimpleDateFormat("dd.MM.yyyy");
return "to_date('" + sdfOutput.format(startDate) + "','DD.MM.YYYY')";
}
/**
* getLastMonday: returns date + given months
* @param today any day
* @return Date with no time
*
* {talendTypes} Date
*
* {Category} TimestampUtil
*
* {param} date today: any day.
*
* {example} getLastMonday(today).
*/
public static java.util.Date getLastMonday(java.util.Date today) {
if (today != null) {
java.util.Calendar c = java.util.Calendar.getInstance(Locale.GERMANY);
c.setTime(today);
c.set(Calendar.DAY_OF_WEEK, Calendar.MONDAY);
c.set(java.util.Calendar.MINUTE, 0);
c.set(java.util.Calendar.SECOND, 0);
c.set(java.util.Calendar.MILLISECOND, 0);
c.set(java.util.Calendar.HOUR_OF_DAY, 0);
return c.getTime();
} else {
return null;
}
}

public static java.util.Date getNextSunday(java.util.Date today) {
if (today != null) {
java.util.Calendar c = java.util.Calendar.getInstance(Locale.GERMANY);
c.setTime(today);
c.set(Calendar.DAY_OF_WEEK, Calendar.SUNDAY);
c.set(java.util.Calendar.MINUTE, 0);
c.set(java.util.Calendar.SECOND, 0);
c.set(java.util.Calendar.MILLISECOND, 0);
c.set(java.util.Calendar.HOUR_OF_DAY, 0);
return c.getTime();
} else {
return null;
}
}
/**
* getLastMonday: returns date + given months
* @param today any day
* @return Date with no time
*
* {talendTypes} Date
*
* {Category} TimestampUtil
*
* {param} date today: any day.
*
* {param} int weekDayIndex: use constants Calendar.MONDAY ... Calendar.SUNDAY
*
* {example} getLastMonday(today).
*/
public static java.util.Date getLastWeekDay(java.util.Date today, int weekDayIndex) {
if (today != null) {
java.util.Calendar c = java.util.Calendar.getInstance(Locale.GERMANY);
c.setTime(today);
if (weekDayIndex > c.get(Calendar.DAY_OF_WEEK)) {
c.add(Calendar.WEEK_OF_YEAR, -1);
}
c.set(Calendar.DAY_OF_WEEK, weekDayIndex);
c.set(java.util.Calendar.MINUTE, 0);
c.set(java.util.Calendar.SECOND, 0);
c.set(java.util.Calendar.MILLISECOND, 0);
c.set(java.util.Calendar.HOUR_OF_DAY, 0);
return c.getTime();
} else {
return null;
}
}

/**
* returns the first not empty date
*
* {Category} TimestampUtil
*
* {talendTypes} Date
*
* {param} date(date1,date2) dates: Date.
*
* {example} coalesce(date1,date2) # "test"
*/
public static java.util.Date coalesce(java.util.Date ...dates) {
for (java.util.Date d : dates) {
if (d != null)
return d;
}
return null;
}
}
One Star

Re: Need to get previous date using getcurrentdate() in tftpget component

Thanks so much for the immediate reply!! Any other way to do it apart from the routines?? We just use the inbuilt system routines. I'm not sure how the customized routines will be migrated to production. Any other way like tjavarow component using java functions in it or using any other components? Thanks!!
Moderator

Re: Need to get previous date using getcurrentdate() in tftpget component

Hi,
Perhaps this article Creating a user routine will point a light to you.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Seven Stars

Re: Need to get previous date using getcurrentdate() in tftpget component

I would have thought your code would give you 2013/05/30 when the current date is 2013/05/01?
Why not just use TalendDate.formatDate("yyyy/MM/dd",TalendDate.addDate(TalendDate.getCurrentDate(),-1,"dd")) or TalendDate.addDate(TalendDate.getDate("yyyy/MM/dd"),"yyyy/MM/dd",-3,"dd")?
One Star

Re: Need to get previous date using getcurrentdate() in tftpget component

hi

i have a job which reads file from a directory and loads into a table.
tFilelist---->tfileinputdelimited----->tmap--->tverticaoutput
the file schema
file id
date
name

so what i want to achieve is compare today's date with "date" column in file if the "date" value is less than current date
i want to update in some table that "date" indicating its old data after the file is completely loaded making status filed in the table as loaded.
this status updation should be on file wise not on each record level in file.