Opened 23 months ago

Last modified 6 months ago

#20546 assigned enhancement

Implement CleanUtils class for common file system operations

Reported by: iwakeh Owned by: metrics-team
Priority: Medium Milestone:
Component: Metrics/CollecTor Version:
Severity: Normal Keywords: metrics-help, metrics-2018
Cc: iwakeh, metrics-team, Samdney Actual Points:
Parent ID: #20518 Points:
Reviewer: Sponsor:

Description

Create a org.torproject.collector.persist.CleanUtils class, which should offer the methods listed below. First create the corresponding org.torproject.collector.persist.CleanUtilsTest class for all these methods. After that make the tests pass.

Use Files.walkFileTree() and SimpleFileVisitor for the implementation.

  1. Basic clean-up method:
/**
 *  All files in the given directory and below are inspected and
 * erased, if they are older than the given cut-off-time.
*/
public static void cleanDir(Path directory, long cutOffTime) {...}
  1. Remove files with a certain ending:
/**
 *  All files in the given directory and below are inspected and
 * erased, if they are older than the given cut-off-time, and their name
 * ends with one of the given patterns.
*/
public static void cleanDirPattern(Path directory, long cutOffTime,
    String ... endingStrings) {...}
  1. Some renaming methods:
/**
 *  All files in the given directory and below are inspected and renamed
 * from <filename><ending> to <fileneame>, if their name ends with the given
 * 'ending'.
*/
public static void renameFiles(Path directory, String ending) {...}


/**
 *  All files in the given directory and below are inspected and renamed
 * from <filename>.tmp to <fileneame>, if their name ends with '.tmp'.
*/
public static void renameTmpFiles(Path directory) {
  renameFiles(directory, ".tmp");
}

Child Tickets

Change History (22)

comment:1 Changed 23 months ago by iwakeh

Keywords: metrics_help added

Excellent ticket for hack-days.

comment:2 Changed 23 months ago by iwakeh

Keywords: metrics-help added; metrics_help removed

comment:3 Changed 22 months ago by aegis2501

Hello, could I help with this?

comment:4 Changed 22 months ago by iwakeh

Great! Help is very much appreciated.

There is a section How to contribute in our wiki.
If you have any questions, just ask here.

Please assign this ticket to your user name when you start working.

Thanks a lot!

comment:5 Changed 22 months ago by aegis2501

Owner: set to aegis2501
Status: newassigned

Thanks! I'm glad to be of help.

Could you perhaps clarify what the contents of CleanUtilsTest should be?

Last edited 22 months ago by aegis2501 (previous) (diff)

comment:6 in reply to:  4 Changed 21 months ago by aegis2501

Replying to iwakeh:

Great! Help is very much appreciated.

There is a section How to contribute in our wiki.
If you have any questions, just ask here.

Please assign this ticket to your user name when you start working.

Thanks a lot!

Hi. I think I'm done: https://github.com/aegis2501/CollecTor

comment:7 Changed 21 months ago by iwakeh

Cool!

I didn't look deeply, e.i. think my way through the code, yet, but it's a very clean style.
Your test coverage is also up to 93% for CleanUtils, great. (I didn't reply to your question
in comment:5, because trac doesn't mail anything for edited comments. So I didn't see the question.)

Maybe, try to think up some tests that trigger exceptions and also to find out, what happens when unexpected input is given, like null or empty Strings or a file disappeared before being erased or never existed etc.

And, we also have a Checkstyle and task ant checks, which currently complains a little about CleanUtils and CleanUtilsTest. Nothing dramatic, mostly indentation and spacing.
Could you get that to pass again?

Thanks a lot for that work!

Last edited 21 months ago by iwakeh (previous) (diff)

comment:8 in reply to:  7 Changed 21 months ago by aegis2501

Replying to iwakeh:

Cool!

I didn't look deeply, e.i. think my way through the code, yet, but it's a very clean style.
Your test coverage is also up to 93% for CleanUtils, great. (I didn't reply to your question
in comment:5, because trac doesn't mail anything for edited comments. So I didn't see the question.)

Maybe, try to think up some tests that trigger exceptions and also to find out, what happens when unexpected input is given, like null or empty Strings or a file disappeared before being erased or never existed etc.

And, we also have a Checkstyle and task ant checks, which currently complains a little about CleanUtils and CleanUtilsTest. Nothing dramatic, mostly indentation and spacing.
Could you get that to pass again?

Thanks a lot for that work!

No problem! I'll work on it.

comment:9 Changed 20 months ago by aegis2501

Additional tests have been added:

https://github.com/aegis2501/CollecTor/blob/master/src/test/java/org/torproject/collector/persist/CleanUtilsTest.java

I've come realize that these additional tests may not be very meaningful however, as they test the implementation of CleanUtils rather than requirements. (Indentation/Spacing has not been changed yet)

Last edited 20 months ago by aegis2501 (previous) (diff)

comment:10 Changed 20 months ago by aegis2501

Resolution: implemented
Status: assignedclosed

comment:11 Changed 20 months ago by iwakeh

Resolution: implemented
Status: closedreopened

Setting to reopen and then needs_review, to prevent this code from being forgotten and follow the usual process.

comment:12 Changed 20 months ago by iwakeh

Status: reopenedneeds_review

comment:13 Changed 19 months ago by iwakeh

Keywords: metrics-help removed
Owner: changed from aegis2501 to iwakeh
Status: needs_reviewassigned

Removed metrics-help tag, as this is already implemented.

Review assigned to myself.

comment:14 Changed 16 months ago by iwakeh

Keywords: metrics-help added
Status: assignedneeds_review

What's open:

  • almost 300 checkstyle complaints (line lengths, indentation, and whitespace mostly)
  • use getFileName method in FileVisitor
  • verify tests and try to adapt them more closely to CollecTor use cases

comment:15 Changed 16 months ago by Samdney

Owner: changed from iwakeh to Samdney
Status: needs_reviewassigned

comment:16 Changed 16 months ago by iwakeh

Cc: iwakeh added

Adding myself to cc to make trac mail updates.

comment:17 Changed 12 months ago by karsten

Summary: implement CleanUtilsImplement CleanUtils class for common file system operations

Let the summary reveal a bit more what this ticket is about.

comment:18 Changed 12 months ago by karsten

Keywords: metrics-2018 added

comment:19 Changed 12 months ago by karsten

Keywords: metrics-2017 added; metrics-2018 removed

comment:20 Changed 9 months ago by iwakeh

Keywords: metrics-2018 added; metrics-2017 removed

Will be completed in 2018.

comment:21 Changed 6 months ago by irl

Cc: metrics-team added

Adding metrics-team to cc

comment:22 Changed 6 months ago by irl

Cc: Samdney added
Owner: changed from Samdney to metrics-team

No activity from Samdney in 11 months on this ticket. Re-assigning to metrics-team.

Note: See TracTickets for help on using tickets.