feat(services/onedrive): List dir shows metadata #5632

erickguan · 2025-02-16T21:22:50Z

Which issue does this PR close?

Part of #4746.

What changes are included in this PR?

Supports OneDrive lister with metadata.

meteorgan

The test workflow in .github/services is missing, we should add one.

meteorgan · 2025-02-20T12:26:44Z

core/src/services/onedrive/lister.rs

-                    oio::Entry::new(&normalized_path, Metadata::new(EntryMode::FILE))
-                }
+            let mut meta = Metadata::new(entry_mode)
+                .with_last_modified(


we have some time functions in chrono_utils.rs, like: parse_datetime_from_rfc2822. Maybe we could reuse them.

Thanks for the tip, added.

Xuanwo · 2025-02-21T02:09:29Z

The test workflow in .github/services is missing, we should add one.

I remembered that we failed to set up a OneDrive test.

erickguan · 2025-02-21T22:08:19Z

I generated a token from a personal test account with Files.ReadWrite. This workflow run.

Additional permissions are useful for API Graph explorer which I recommend:

DelegatedPermissionGrant.ReadWrite.All
Directory.Read.All

What do you want me next? These actions look sensible:

Merge this PR as is.
Set up an OpenDAL test account.
Fix bugs in a new PR.

meteorgan · 2025-02-22T09:37:41Z

I generated a token from a personal test account with Files.ReadWrite. This workflow run.

Additional permissions are useful for API Graph explorer which I recommend:

DelegatedPermissionGrant.ReadWrite.All

Directory.Read.All

What do you want me next? These actions look sensible:

Merge this PR as is.

Set up an OpenDAL test account.

Fix bugs in a new PR.

Since we don't have a test workflow for this, have you run the tests locally ? You can use the command OPENDAL_TEST=onedrive cargo test behavior --features tests,services-onedrive -- --show-output. if all the tests pass, I think we can proceed with merging this PR.

Most my test folders have 0 in size, but one has 3. I didn't dig into it.

erickguan · 2025-02-25T20:56:54Z

Rebased and ready.

OPENDAL_TEST=onedrive cargo test behavior --features tests,services-onedrive -- --show-output

   Compiling opendal v0.52.0 (/home/erickg/Dev/opendal/core)
    Finished `test` profile [unoptimized + debuginfo] target(s) in 11.49s
     Running unittests src/lib.rs (target/debug/deps/opendal-9022af4dd162386a)

running 0 tests

successes:

successes:

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 134 filtered out; finished in 0.00s

     Running tests/behavior/main.rs (target/debug/deps/behavior-c8073076fdb6f2e9)

running 102 tests
test behavior::test_delete_with_version                       ... ok
test behavior::test_delete_with_not_existing_version          ... ok
test behavior::test_list_with_start_after                     ... ok
test behavior::test_list_with_versions_and_start_after        ... ok
test behavior::test_reader_with_if_modified_since             ... ok
test behavior::test_reader_with_if_unmodified_since           ... ok
test behavior::test_reader_with_if_none_match                 ... ok
test behavior::test_read_with_if_match                        ... ok
test behavior::test_read_with_if_none_match                   ... ok
test behavior::test_read_with_if_modified_since               ... ok
test behavior::test_read_with_if_unmodified_since             ... ok
test behavior::test_list_files_with_versions                  ... ok
test behavior::test_list_files_with_deleted                   ... ok
test behavior::test_list_with_versions_and_limit              ... ok
test behavior::test_reader_with_if_match                      ... ok
test behavior::test_read_with_override_content_type           ... ok
test behavior::test_read_with_version                         ... ok
test behavior::test_read_with_not_existing_version            ... ok
test behavior::test_read_with_override_content_disposition    ... ok
test behavior::test_read_with_override_cache_control          ... ok
test behavior::test_read_not_exist                            ... ok
test behavior::test_check                                     ... ok
test behavior::test_list_non_exist_dir                        ... ok
test behavior::test_stat_with_if_match                        ... ok
test behavior::test_stat_with_if_none_match                   ... ok
test behavior::test_stat_with_if_modified_since               ... ok
test behavior::test_stat_with_if_unmodified_since             ... ok
test behavior::test_stat_with_override_cache_control          ... ok
test behavior::test_stat_with_override_content_disposition    ... ok
test behavior::test_stat_with_override_content_type           ... ok
test behavior::test_stat_root                                 ... ok
test behavior::test_stat_with_version                         ... ok
test behavior::stat_with_not_existing_version                 ... ok
test behavior::test_list_root_with_recursive                  ... ok
test behavior::test_write_with_empty_content                  ... ok
test behavior::test_write_with_dir_path                       ... ok
test behavior::test_delete_not_existing                       ... ok
test behavior::test_write_with_cache_control                  ... ok
test behavior::test_write_with_content_type                   ... ok
test behavior::test_write_with_content_disposition            ... ok
test behavior::test_write_with_content_encoding               ... ok
test behavior::test_write_with_if_none_match                  ... ok
test behavior::test_write_with_if_not_exists                  ... ok
test behavior::test_write_with_if_match                       ... ok
test behavior::test_write_with_user_metadata                  ... ok
test behavior::test_read_with_dir_path                        ... ok
test behavior::test_writer_write                              ... ok
test behavior::test_stat_not_exist                            ... ok
test behavior::test_writer_write_with_concurrent              ... ok
test behavior::test_writer_sink                               ... ok
test behavior::test_writer_sink_with_concurrent               ... ok
test behavior::test_writer_abort                              ... ok
test behavior::test_create_dir                                ... ok
test behavior::test_writer_futures_copy                       ... ok
test behavior::test_writer_futures_copy_with_concurrent       ... ok
test behavior::test_writer_return_metadata                    ... ok
test behavior::test_stat_dir                                  ... ok
test behavior::test_delete_empty_dir                          ... ok
test behavior::test_writer_abort_with_concurrent              ... ok
test behavior::test_create_dir_existing                       ... ok
test behavior::test_blocking_create_dir                       ... ok
test behavior::test_list_sub_dir                              ... ok
test behavior::test_blocking_read_not_exist                   ... ok
test behavior::test_delete_file                               ... ok
test behavior::test_stat_with_special_chars                   ... ok
test behavior::test_read_range                                ... ok
test behavior::test_delete_with_special_chars                 ... ok
test behavior::test_write_returns_metadata                    ... ok
test behavior::test_blocking_create_dir_existing              ... ok
test behavior::test_list_prefix                               ... ok
test behavior::test_stat_nested_parent_dir                    ... ok
test behavior::test_blocking_stat_not_exist                   ... ok
test behavior::test_blocking_write_with_dir_path              ... ok
test behavior::test_write_only                                ... ok
test behavior::test_blocking_stat_dir                         ... ok
test behavior::test_list_dir_with_file_path                   ... ok
test behavior::test_read_full                                 ... ok
test behavior::test_stat_file                                 ... ok
test behavior::test_list_file_with_recursive                  ... ok
test behavior::test_write_with_special_chars                  ... ok
test behavior::test_stat_not_cleaned_path                     ... ok
test behavior::test_read_with_special_chars                   ... ok
test behavior::test_list_dir                                  ... ok
test behavior::test_blocking_delete_file                      ... ok
test behavior::test_blocking_write_file                       ... ok
test behavior::test_blocking_stat_with_special_chars          ... ok
test behavior::test_blocking_remove_one_file                  ... ok
test behavior::test_blocking_read_range                       ... ok
test behavior::test_blocking_stat_file                        ... ok
test behavior::test_reader                                    ... ok
test behavior::test_blocking_write_with_special_chars         ... ok
test behavior::test_remove_one_file                           ... ok
test behavior::test_blocking_write_returns_metadata           ... ok
test behavior::test_blocking_read_full                        ... ok
test behavior::test_list_dir_with_recursive_no_trailing_slash ... ok
test behavior::test_list_nested_dir                           ... ok
test behavior::test_writer_write_with_overwrite               ... ok
test behavior::test_list_dir_with_recursive                   ... ok
test behavior::test_list_empty_dir                            ... ok
test behavior::test_remove_all                                ... ok
test behavior::test_list_rich_dir                             ... ok
test behavior::test_delete_stream                             ... ok

test result: ok. 102 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 38.88s

meteorgan · 2025-02-26T04:41:52Z

LGTM. Thanks for working on this. @erickguan

erickguan · 2025-02-26T08:54:12Z

@meteorgan happy to help!

Xuanwo

Thank you @erickguan for working on this and thank you @meteorgan for the review.

Xuanwo · 2025-02-26T10:18:09Z

By the way, @erickguan would you like to help fix the conflicts?

erickguan · 2025-02-26T11:38:12Z

Sure, I will rebase it and ping you.

erickguan requested a review from Xuanwo as a code owner February 16, 2025 21:22

github-actions bot added the releases-note/feat label Feb 16, 2025

meteorgan reviewed Feb 20, 2025

View reviewed changes

erickguan force-pushed the onedrive-metadata branch 3 times, most recently from 26c9d63 to 3e64864 Compare February 21, 2025 21:43

erickguan force-pushed the onedrive-metadata branch from 3e64864 to db4c88b Compare February 21, 2025 22:08

This was referenced Feb 23, 2025

fix(services/onedrive): fix OneDrive behavior tests #5652

Merged

Set up behavior tests for OneDrive #5653

Open

feat(services/onedrive): List dir shows metadata

f370374

Most my test folders have 0 in size, but one has 3. I didn't dig into it.

erickguan force-pushed the onedrive-metadata branch from db4c88b to f370374 Compare February 25, 2025 20:56

erickguan requested a review from meteorgan February 25, 2025 20:57

meteorgan approved these changes Feb 26, 2025

View reviewed changes

Xuanwo approved these changes Feb 26, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(services/onedrive): List dir shows metadata #5632

feat(services/onedrive): List dir shows metadata #5632

erickguan commented Feb 16, 2025

meteorgan left a comment

meteorgan Feb 20, 2025

erickguan Feb 21, 2025

Xuanwo commented Feb 21, 2025

erickguan commented Feb 21, 2025 •

edited

Loading

meteorgan commented Feb 22, 2025 •

edited

Loading

erickguan commented Feb 25, 2025

meteorgan commented Feb 26, 2025 •

edited

Loading

erickguan commented Feb 26, 2025

Xuanwo left a comment

Xuanwo commented Feb 26, 2025

erickguan commented Feb 26, 2025

feat(services/onedrive): List dir shows metadata #5632

Are you sure you want to change the base?

feat(services/onedrive): List dir shows metadata #5632

Conversation

erickguan commented Feb 16, 2025

Which issue does this PR close?

What changes are included in this PR?

meteorgan left a comment

Choose a reason for hiding this comment

meteorgan Feb 20, 2025

Choose a reason for hiding this comment

erickguan Feb 21, 2025

Choose a reason for hiding this comment

Xuanwo commented Feb 21, 2025

erickguan commented Feb 21, 2025 • edited Loading

meteorgan commented Feb 22, 2025 • edited Loading

erickguan commented Feb 25, 2025

meteorgan commented Feb 26, 2025 • edited Loading

erickguan commented Feb 26, 2025

Xuanwo left a comment

Choose a reason for hiding this comment

Xuanwo commented Feb 26, 2025

erickguan commented Feb 26, 2025

erickguan commented Feb 21, 2025 •

edited

Loading

meteorgan commented Feb 22, 2025 •

edited

Loading

meteorgan commented Feb 26, 2025 •

edited

Loading