Uses syntax:

contents:
  foo:
    "*.baz"   # Case insensitive.  All files are sub-nodes of 'foo'

A few notes:

• case insensitivity was actually kindof a pain -- but it works now
• ~~there are a couple tools in tensorflow that I didn't bring over -- like the backported pathlib in tools.compat, and file->node duplicate naming conflict resolver.~~ These have been brought over. There may be conflicts for the TensorFlow branch once this is merged into master..
• subdirs are made into nodenames for now instead of making subnodes -- so subdir_foo_csv
  • This now uses only the filename of the found file, and appends a number if there's a conflict.
• ~~still rough~~

Normally I'd just do a try: except: block on or in main(), but at load-time, we load a bunch of external modules that take a lot of time, during which ctrl-c will cause an exception that misses that block. ..so, this was added to quilt/__init__.py.

Also, sometimes during development we want to be able to see the traceback -- for example, if we're wondering what's taking so long, or what function is causing network activity. Toward that end, there's now a variable quilt._DEV_MODE which enables/disables traceback.

If your first argument is --dev, or if the environment has QUILT_DEV_MODE=True, tracebacks will still be shown. Help for --dev is suppressed and doesn't show up in quilt help, quilt --help, etc.

I have a multi-user Linux server that I will be using for a Data Science class. I would like to use quilt to distribute datasets with Jupyter Notebooks to students. But I don't want all students to download their own copies of the data when using quilt. I see that quilt uses appdirs.user_data_dir() to get the directory to use, and that I can set XDG_DATA_HOME to override that location:

https://github.com/ActiveState/appdirs/blob/master/appdirs.py#L92

• Will this break quilt?
• Will this give me the optimization of each unique dataset only being downloaded a single time for all users?
• How will this affect users creating and pushing datasets?

Thanks!

Adds pagination to the package list. WIP.

TODO:

• [x] styling tweaks + use proper colors and sizes from the design system or smth
• [x] integrate pagination into profile view
• [x] intl
• [x] unit tests

Open questions:

• [x] is UX okayish? (wording, results-per-page selection)
• [x] where to store pagination state? now the component is a "controlled component" with its state in redux, but maybe it makes sense to use local state instead (it will make it easier to use)
  • locally
• [x] should we make the PackageList paginated by default?
  • yes
• [x] do we need to link url params to the pagination state?
  • no

cc @asah

TODO

• [x] wire up member actions (remove, reset password)
• [x] indicate activity while performing member actions (lock, spinner)
• [x] wire up package actions (delete)
• [x] indicate activity while performing package actions (lock, spinner)
• [x] handle member addition
• [x] styling tweaks
• [x] overall polish
• [ ] pagination?
• [x] proper i18n?
• [x] fix npm run build
• [x] fix linter warnings
• [x] confirmations for actions?

Show traffic stats on the package page.

Depends on:

• ~#587~
• ~#588~
• #594

TODO

• [x] tweak styling
• [x] intl
• [x] wire up the api
• [x] fill up the sparkline timeseries with zeroes to the fixed length (52)? @akarve
  • will be done on the API side
• [x] get total installs count somewhere or change display logic to not show it altogether @akarve
  • will be done on the API side
• [x] wait for the API changes and adjust if necessary

In order to simplify access to the config data throughout the app (to a simple import cfg from 'utils/Config') and setup of some tools (such as sentry), I've decided to refactor the config system to use a synchronous JS file which sets a global variable that later can be accessed by the app code.

This requires a change in deployment which will come soon (later today).

UPD: Actually, deployment change is only relevant for OPEN, so we can merge this PR without waiting for that change.

In order for local catalog instance to continue functioning, one should rename config.json file in static-dev folder to config.js and prepend the contents with window.QUILT_CATALOG_CONFIG =

TODO

• [x] changelog

Hi team,

thanks for the great work on this project. With the current behavior when users have tabular data like CSV, its gets converted to parquet upon quilt push. But, when the data are already stored as a series of parquet files, the binaries are simply uploaded to the quilt server and the underlying dataframes aren't registered under tables/

is it possible to allow uploading of parquet files directly so that the dataframes are registered and useable?

Summary

Includes quilt export and command.export(...)

Only handles FileNode objects, not TableNode objects (minimum for TF support).

Depends On

~~#536 Minor test improvements~~ ~~#537 Build.py uncaught exception fix, other minor misc from export PR~~

• highlight.js: 10.7.2 → 11.0.1
• katex: 0.13.5 → 0.13.11
• react-ace: 9.4.0 → 9.4.1
• remarkable: 1.7.4 → 2.0.1
• sanitize.css: 11.0.1 → 12.0.1
• vega-embed: 6.17.0 → 6.18.2

Description

quilt3 fails to load config on Windows as of 3.1.12, giving me this error on import quilt3:

AttributeError: 'WindowsPath' object has no attribute 'read'

from inside the yaml code. Tracing around, I believe the root is that this check should accept all pathlib.Paths, not just pathlib.PosixPath. Without that check, quilt3 passes a pathlib object instead of a stream into yaml.safe_load, which triggers the above error.

Changing this worked for me locally.

Bumps json5 from 1.0.1 to 1.0.2.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Description

TODO

• [ ] Unit tests
• [ ] Automated tests (e.g. Preflight)
• [ ] Documentation
  • [ ] Python: Run build.py for new docstrings
  • [ ] JavaScript: basic explanation and screenshot of new features
• [ ] Changelog entry (skip if change is not significant to end users, e.g. docs only)

Description

• Documentation to describe how to get notifications for cross-account S3 bucket changes
• All PNG screenshot images have private data blurred and have been crushed with ImageOptim

Screencast from 2022-12-29 22-58-55.webm

Example: /b/fiskus-sandbox-dev/packages/fiskus/perspective-with-images/tree/latest/sample.csv

• [ ] Changelog entry (skip if change is not significant to end users, e.g. docs only)

This PR contains the following updates:

| Package | Change | |---|---| | jsonwebtoken | 8.5.1 -> 9.0.0 |

GitHub Vulnerability Alerts

CVE-2022-23540

Overview

In versions <=8.5.1 of jsonwebtoken library, lack of algorithm definition in the jwt.verify() function can lead to signature validation bypass due to defaulting to the none algorithm for signature verification.

Am I affected?

You will be affected if you do not specify algorithms in the jwt.verify() function

How do I fix it?

Update to version 9.0.0 which removes the default support for the none algorithm in the jwt.verify() method.

Will the fix impact my users?

There will be no impact, if you update to version 9.0.0 and you don’t need to allow for the none algorithm. If you need 'none' algorithm, you have to explicitly specify that in jwt.verify() options.

CVE-2022-23541

Overview

Versions <=8.5.1 of jsonwebtoken library can be misconfigured so that passing a poorly implemented key retrieval function (referring to the secretOrPublicKey argument from the readme link) will result in incorrect verification of tokens. There is a possibility of using a different algorithm and key combination in verification than the one that was used to sign the tokens. Specifically, tokens signed with an asymmetric public key could be verified with a symmetric HS256 algorithm. This can lead to successful validation of forged tokens.

Am I affected?

You will be affected if your application is supporting usage of both symmetric key and asymmetric key in jwt.verify() implementation with the same key retrieval function.

How do I fix it?

Update to version 9.0.0.

Will the fix impact my users?

There is no impact for end users

CVE-2022-23539

Overview

Versions <=8.5.1 of jsonwebtoken library could be misconfigured so that legacy, insecure key types are used for signature verification. For example, DSA keys could be used with the RS256 algorithm.

Am I affected?

You are affected if you are using an algorithm and a key type other than the combinations mentioned below

| Key type | algorithm | |----------|------------------------------------------| | ec | ES256, ES384, ES512 | | rsa | RS256, RS384, RS512, PS256, PS384, PS512 | | rsa-pss | PS256, PS384, PS512 |

And for Elliptic Curve algorithms:

| alg | Curve | |-------|------------| | ES256 | prime256v1 | | ES384 | secp384r1 | | ES512 | secp521r1 |

How do I fix it?

Update to version 9.0.0. This version validates for asymmetric key type and algorithm combinations. Please refer to the above mentioned algorithm / key type combinations for the valid secure configuration. After updating to version 9.0.0, If you still intend to continue with signing or verifying tokens using invalid key type/algorithm value combinations, you’ll need to set the allowInvalidAsymmetricKeyTypes option to true in the sign() and/or verify() functions.

Will the fix impact my users?

There will be no impact, if you update to version 9.0.0 and you already use a valid secure combination of key type and algorithm. Otherwise, use the allowInvalidAsymmetricKeyTypes option to true in the sign() and verify() functions to continue usage of invalid key type/algorithm combination in 9.0.0 for legacy compatibility.

CVE-2022-23529

Overview

For versions <=8.5.1 of jsonwebtoken library, if a malicious actor has the ability to modify the key retrieval parameter (referring to the secretOrPublicKey argument from the readme link) of the jwt.verify() function, they can gain remote code execution (RCE).

Am I affected?

You are affected only if you allow untrusted entities to modify the key retrieval parameter of the jwt.verify() on a host that you control.

How do I fix it?

Update to version 9.0.0

Will the fix impact my users?

The fix has no impact on end users.

Credits

Palo Alto Networks

Configuration

📅 Schedule: Branch creation - "" (UTC), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

• [ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate. View repository job log here.