8000 Add functionality to return parent folder that has files been modified · Issue #66 · Sage-Bionetworks/synapsemonitor · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Add functionality to return parent folder that has files been modified #66

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
danlu1 opened this issue Mar 7, 2022 · 11 comments
Open

Comments

@danlu1
Copy link
danlu1 commented Mar 7, 2022

I'm not sure if there is a functionality already, but I found if I create a fileview and only include folder in its scope. The view is not changing the modifiedOn value when a file is updated/uploaded to the scoped folders but it is changing when a subfolder is added. If we are using files in the view scope, the modifiedOn value reflects the most recent changes but would send us hundreds of SynIDs that would be less informative than only giving us the folders that have content changed.

@hhunterzinck
Copy link
Collaborator

The only currently implemented functionality for detecting modified entities relies on the timestamp of modification associated with the entity itself, not its contents. Flagging folder entity types as modified based on their modified contents could be an interesting idea for expanding functionality to different entity types

Linked issue #27

@thomasyu888
Copy link
Member
thomasyu888 commented Mar 8, 2022

@danlu1 , @hhunterzinck . Currently the feature that Dan is requesting doesn't exist in Synapse. I think we could make a good use case to include this feature. Currently when you upload something into a folder, the folder itself on Synapse doesn't change in modifiedOn date. This does not mimic that of the unix system: when you add or modify something within a folder on unix, it will mark the folder as modified.

I think this feature is out of scope from synapsemonitor, but we could open a JIRA ticket to try to get this added to the platform. Thoughts?

@hhunterzinck
Copy link
Collaborator

@thomasyu888 Monitoring contents rather than modifiedOn timestamp would definitely be a break in the implementation practice for synapsemonitor. What's interesting is that while the modifiedOn timestamp doesn't change for projects when file or folder entities are added, the Your Projects page in the web UI sorts projects by the Last Activity On, which does encapsulate added file or folder contents. Is there a way to access the last activity timestamp through the synapse python client?

@thomasyu888
Copy link
Member

@hhunterzinck Oh interesting - I'm not sure this is provided by the the GET /entity endpoints. It is however provided by this endpoint, which is the one you were talking about. if you take a look at the response object, it contains a lastActivity which I can't find elsewhere in the API.

Interestingly, it also isn't part of the Project response object. This would have to be a feature that would need to be added to entities.

@hhunterzinck
Copy link
Collaborator

@thomasyu888 Thanks for the links. I guess since the lastActivity timestamp isn't returned for entities in general, that approach wouldn't work for @danlu1 's use case. And unless there is a particular reason to only alter the modifiedOn timestamp for folder entities when things like annotations rather than contents are modified, then I'd support a Jira ticket to alter synapse behavior accordingly to make consistent with linux/unix.

If the change is made in Synapse and synapsemonitor expands functionality to allow the user to specify entity types, then a user could specify the synapse ID of a folder and restrict to monitoring Folder entity types to satisfy the requested functionality.

@danlu1
Copy link
Author
danlu1 commented Mar 8, 2022

Thanks @thomasyu888 @hhunterzinck! I think it would be better if we only alter the modifiedOn timestamp for folder entities when contents are modified (i.e. file's content changes, new file has been added) rather than annotations are added. For my use case, I'm thinking abt adapting https://github.com/thomasyu888/synapse-monitor and tweaking it to crawl through records in fileview.

@thomasyu888
Copy link
Member
thomasyu888 commented Mar 16, 2022

@danlu1 what feature would you be using in the old synapse-monitor fork? Is it the wiki generation code?

To unblock you, what do you think of using fileviews? You could create a fileview for that particular folder and run a query statement to see if there have been any changes in the past X number of days.

@danlu1
Copy link
Author
danlu1 commented Mar 16, 2022

@thomasyu888 since my goal is to check new files have been added to a bunch of staging folders in the past X number of days. I'm thinking that I can adopt the fork and use it for fileview with scope of those staging folders.

@thomasyu888
Copy link
Member
thomasyu888 commented Mar 16, 2022

@danlu1 I think you can achieve what you want this this particular repository. I would move away from using my old fork as that code is ultimately unmaintained.

@danlu1
Copy link
Author
danlu1 commented Mar 16, 2022

@thomasyu888 If I understand correctly, the current functionalities checks 'modifiedOn' values for records in the fileview. Did I miss anything?

@thomasyu888
Copy link
Member
thomasyu888 commented Mar 16, 2022

Ah, you want to check the "createdOn" date? Sorry about that! ya - I think i have that as a ticket to support.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants
0