summaryrefslogtreecommitdiff
path: root/.github
AgeCommit message (Collapse)Author
2024-09-06More valgrind fixes (#12990)Peter Dillinger
Summary: * https://github.com/facebook/rocksdb/issues/12936 was insufficient to fix the std::optional false positives. Making a fix validated in CI this time (see https://github.com/facebook/rocksdb/issues/12991) * valgrind grinds to a halt on startup on my dev machine apparently because it expects internet access. Disable its attempts to access the internet when git is using a proxy. * Move PORTABLE=1 from CI job to the Makefile. Without it, valgrind complains about illegal instructions (too new) Pull Request resolved: https://github.com/facebook/rocksdb/pull/12990 Test Plan: manual, watch nightly valgrind job Reviewed By: ltamasi Differential Revision: D62203242 Pulled By: pdillinger fbshipit-source-id: a611b08da7dbd173b0709ed7feb0578729553a17
2024-08-26Scope down workflow permissions (#12973)Jay Huh
Summary: Followed instruction per https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#defining-access-for-the-github_token-scopes It turns out that we did not need any of these except `Metadata: read`. Before ``` GITHUB_TOKEN Permissions Actions: write Attestations: write Checks: write Contents: write Deployments: write Discussions: write Issues: write Metadata: read Packages: write Pages: write PullRequests: write RepositoryProjects: write SecurityEvents: write Statuses: write ``` After ``` GITHUB_TOKEN Permissions Metadata: read ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/12973 Test Plan: GitHub Actions triggered by this PR Reviewed By: cbi42 Differential Revision: D61812651 Pulled By: jaykorean fbshipit-source-id: 4413756c93f503e8b2fb77eb8b684ef9e6a6c13d
2024-08-23Disable benchmark-linux (#12964)Jay Huh
Summary: Disabling the job temporarily. We will re-enable this when ready again Pull Request resolved: https://github.com/facebook/rocksdb/pull/12964 Reviewed By: ltamasi Differential Revision: D61740941 Pulled By: jaykorean fbshipit-source-id: 167e50c4f5e38d508a8e56633261611467f30690
2024-08-20Enable Continuous Benchmarking of RocksDB (#12885)Radek Hubner
Summary: This pull request transitions the benchmarking process from CircleCI to GitHub Actions. The benchmarking jobs will now be executed on a self-hosted runner. Unlike the previous CircleCI configuration, where jobs were queued due to the long execution time (nearly 60 minutes per job), the new setup schedules the benchmarking tasks to run every two hours. Closes https://github.com/facebook/rocksdb/issues/12615 Pull Request resolved: https://github.com/facebook/rocksdb/pull/12885 Reviewed By: pdillinger Differential Revision: D61422468 Pulled By: jaykorean fbshipit-source-id: 10535865c849797825f9652e4e9ef367b3d73599
2024-08-01Attempt to fix the nightly build-linux-clang-13-asan-ubsan-with-folly buildLevi Tamasi
Summary: https://github.com/facebook/rocksdb/pull/12801 updated the version of `folly` used in RocksDB builds to a revision that requires `g++` version 10 when built with a GNU toolchain. This shouldn't really matter for this nightly GitHub Actions job, since we're supposed to be building with `clang++-13`; however, due to the way the compilers had been set, seems like we were historically only building RocksDB with `clang` (and `folly` with `gcc-9`, which led to a broken build after the update). Attempt to fix this by setting `CC` / `CXX` to `clang` / `clang++` in the job's environment. Reviewed By: pdillinger Differential Revision: D60534452 fbshipit-source-id: c7b5a02409fb1ea50e4524731237f7bc8d3f7ca6
2024-06-22Fix folly build (#12795)Andrew Kryczka
Summary: - Updated pinned folly version to the latest - gcc/g++ 10 is required since https://github.com/facebook/folly/commit/2c1c617e9e so we had to modify the tests using gcc/g++ 7 - libsodium 1.0.17 is no longer downloadable from GitHub so I found it elsewhere. I will submit a PR for that upstream to folly - USE_FOLLY_LITE changes - added boost header dependency instead of commenting out the `#include`s since that approach stopped working - added "folly/lang/Exception.cpp" to the compilation Pull Request resolved: https://github.com/facebook/rocksdb/pull/12795 Reviewed By: hx235 Differential Revision: D58916693 Pulled By: ajkr fbshipit-source-id: b5f9bca2d929825846ac898b785972b071db62b1
2024-04-22convert circleci arm jobs to github actions (#12569)Rob Anderson
Summary: This pull request converts the CircleCI jobs that run on ARM runners to GitHub actions jobs. With this change you can retire the [circleci config](https://github.com/facebook/rocksdb/blob/main/.circleci/config.yml) for this repo. This change assumes you have [ARM runners](https://github.blog/changelog/2023-10-30-accelerate-your-ci-cd-with-arm-based-hosted-runners-in-github-actions/) with the label `4-core-ubuntu-arm`. --- [Here is a workflow run in my fork showing these jobs passing](https://github.com/robandpdx-org/rocksdb/actions/runs/8760406181/job/24045304383). --- https://fburl.com/workplace/f6mz6tmw Pull Request resolved: https://github.com/facebook/rocksdb/pull/12569 Reviewed By: ltamasi Differential Revision: D56435439 Pulled By: ajkr fbshipit-source-id: a24d79f21baca01beda232746f90b2853f27a664
2024-03-14Fix windows build and CI (#12426)Peter Dillinger
Summary: Issue https://github.com/facebook/rocksdb/issues/12421 describes a regression in the migration from CircleCI to GitHub Actions in which failing build steps no longer fail Windows CI jobs. In GHA with pwsh (new preferred powershell command), only the last non-builtin command (or something like that) affects the overall success/failure result, and failures in external commands do not exit the script, even with `$ErrorActionPreference = 'Stop'` and `$PSNativeCommandErrorActionPreference = $true`. Switching to `powershell` causes some obscure failure (not seen in CircleCI) about the `-Lo` option to `curl`. Here we work around this using the only reasonable-but-ugly way known: explicitly check the result after every non-trivial build step. This leaves us highly susceptible to future regressions with unchecked build steps in the future, but a clean solution is not known. This change also fixes the build errors that were allowed to creep in because of the CI regression. Also decreased the unnecessarily long running time of DBWriteTest.WriteThreadWaitNanosCounter. For background, this problem explicitly contradicts GitHub's documentation, and GitHub has known about the problem for more than a year, with no evidence of caring or intending to fix. https://github.com/actions/runner-images/issues/6668 Somehow CircleCI doesn't have this problem. And even though cmd.exe and powershell have been perpetuating DOS-isms for decades, they still seem to be a somewhat active "hot mess" when it comes to sensible, consistent, and documented behavior. Fixes https://github.com/facebook/rocksdb/issues/12421 A history of some things I tried in development is here: https://github.com/facebook/rocksdb/compare/main...pdillinger:rocksdb:debug_windows_ci_orig Pull Request resolved: https://github.com/facebook/rocksdb/pull/12426 Test Plan: CI, including https://github.com/facebook/rocksdb/issues/12434 where I have temporarily enabled other Windows builds on PR with this change Reviewed By: cbi42 Differential Revision: D54903698 Pulled By: pdillinger fbshipit-source-id: 116bcbebbbf98f347c7cf7dfdeebeaaed7f76827
2024-01-31Enable io_uring in stress test (#12313)Akanksha Mahajan
Summary: Enable io_uring in stress test Pull Request resolved: https://github.com/facebook/rocksdb/pull/12313 Test Plan: Crash test Reviewed By: anand1976 Differential Revision: D53238319 Pulled By: akankshamahajan15 fbshipit-source-id: c0c8e6a6479f6977210370606e9d551c1299ba62
2024-01-29Add more detail to some statuses (#12307)Peter Dillinger
Summary: and also fix comment/label on some MacOS CI jobs. Motivated by a crash test failure missing a definitive indicator of the genesis of the status: ``` file ingestion error: Operation failed. Try again.: ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/12307 Test Plan: just cosmetic changes. These statuses should not arise frequently enough to be a performance issue (copying messages). Reviewed By: jaykorean Differential Revision: D53199529 Pulled By: pdillinger fbshipit-source-id: ad83daaa5d80f75c9f81158e90fb6d9ecca33fe3
2024-01-19Use the correct Docker Image for RocksJava on Linux (#12169)Adam Retter
Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/12169 Reviewed By: pdillinger Differential Revision: D52715225 Pulled By: ajkr fbshipit-source-id: 28476d363034fa1bb9c8c919d577c03b6391451b
2023-12-28Disable GitHub Actions jobs on forks (#12191)Peter Dillinger
Summary: See new comment in pr-jobs.yml for context. I tried avoiding the massive copy-paste through some trial and error in https://github.com/facebook/rocksdb/issues/12156, but was unsuccessful. Also upgrading actions/setup-python to v5 to fix a warning. Pull Request resolved: https://github.com/facebook/rocksdb/pull/12191 Test Plan: Here's an example of a *bad* run from my fork, prior to this change: https://github.com/pdillinger/rocksdb/actions/runs/7303363126 Here's the "skipped" run associated with this change on my fork: https://github.com/pdillinger/rocksdb/actions/runs/7352251207 Here's the actual run associated with this PR: https://github.com/facebook/rocksdb/actions/runs/7352262420 Reviewed By: ajkr Differential Revision: D52451292 Pulled By: pdillinger fbshipit-source-id: 9e0d3db8a40e3257e6f912a5cba72de76f4827fa
2023-12-21Initial CircleCI -> GitHub Actions migration (#12163)Peter Dillinger
Summary: * Largely based on https://github.com/facebook/rocksdb/issues/12085 but grouped into one large workflow because of bad GHA UI design (see comments). * Windows job details consolidated into an action file so that those jobs can easily move between per-pr-push and nightly. * Simplify some handling of "CIRCLECI" environment and add "GITHUB_ACTIONS" in the same places * For jobs that we want to go in pr-jobs or nightly there are disabled "candidate" workflows with draft versions of those jobs. * ARM jobs are disabled waiting on full GHA support. * build-linux-java-static needed some special attention to work, due to GLIBC compatibility issues (see comments). Pull Request resolved: https://github.com/facebook/rocksdb/pull/12163 Test Plan: Nightly jobs can be seen passing between these two links: https://github.com/facebook/rocksdb/actions/runs/7266835435/job/19799390061?pr=12163 https://github.com/facebook/rocksdb/actions/runs/7269697823/job/19807724471?pr=12163 And per-PR jobs of course passing on this PR. Reviewed By: hx235 Differential Revision: D52335810 Pulled By: pdillinger fbshipit-source-id: bbb95196f33eabad8cddf3c6b52f4413c80e034d
2023-06-13Fix failed CI job "Check buck targets and code format" (#11532)Changyu Bi
Summary: the first CI step "Check buck targets and code format..." is failing with the following error message: ``` Download action repository 'wei/wget@v1' (SHA:c15e476d1463f4936cb54f882170d9d631f1aba5) Error: An action could not be found at the URI 'https://api.github.com/repos/wei/wget/tarball/c15e476d1463f4936cb54f882170d9d631f1aba5' ``` Not sure why the action is lost, but it seems we can use wget directly. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11532 Test Plan: watch CI job "Check buck targets and code format" passes Reviewed By: ltamasi Differential Revision: D46700626 Pulled By: cbi42 fbshipit-source-id: 53c09f27965444b533b3fe3755aec922571bba2c
2022-10-04ci: add GitHub token permissions for workflow (#10549)Varun Sharma
Summary: This PR adds minimum token permissions for the GITHUB_TOKEN in GitHub Actions workflows using https://github.com/step-security/secure-workflows. GitHub recommends defining minimum GITHUB_TOKEN permissions for securing GitHub Actions workflows - https://github.blog/changelog/2021-04-20-github-actions-control-permissions-for-github_token/ - https://docs.github.com/en/actions/security-guides/automatic-token-authentication#modifying-the-permissions-for-the-github_token - The Open Source Security Foundation (OpenSSF) [Scorecards](https://github.com/ossf/scorecard) treats not setting token permissions as a high-risk issue This project is part of the top 100 critical projects as per OpenSSF (https://github.com/ossf/wg-securing-critical-projects), so fixing the token permissions to improve security. Before the change: `GITHUB_TOKEN` has `write` permissions for multiple scopes, e.g. https://github.com/facebook/rocksdb/runs/7936368166?check_suite_focus=true#step:1:19 After the change: `GITHUB_TOKEN` will have minimum permissions needed for the jobs. Signed-off-by: Varun Sharma <varunsh@stepsecurity.io> Pull Request resolved: https://github.com/facebook/rocksdb/pull/10549 Reviewed By: ajkr Differential Revision: D38923184 Pulled By: jay-zhuang fbshipit-source-id: 0c48f98fe90665e53724f57a7d3b01dd80f34a93
2022-03-01Use released clang-format instead of the one from dev branch (#9646)Jay Zhuang
Summary: We should use the released clang-format version instead of the one from dev branch. Otherwise the format report could be inconsistent with local development env and CI. e.g.: https://github.com/facebook/rocksdb/issues/9644 Pull Request resolved: https://github.com/facebook/rocksdb/pull/9646 Test Plan: CI Reviewed By: riversand963 Differential Revision: D34554065 Pulled By: jay-zhuang fbshipit-source-id: b841bc400becb4272be18c803eb03a7a1172da6f
2021-09-07Add (& fix) some simple source code checks (#8821)Peter Dillinger
Summary: * Don't hardcode namespace rocksdb (use ROCKSDB_NAMESPACE) * Don't #include <rocksdb/...> (use double quotes) * Support putting NOCOMMIT (any case) in source code that should not be committed/pushed in current state. These will be run with `make check` and in GitHub actions Pull Request resolved: https://github.com/facebook/rocksdb/pull/8821 Test Plan: existing tests, manually try out new checks Reviewed By: zhichao-cao Differential Revision: D30791726 Pulled By: pdillinger fbshipit-source-id: 399c883f312be24d9e55c58951d4013e18429d92
2021-02-09Update clang-format-diff.py path (#7944)Yanqin Jin
Summary: Recent Github actions of format checking fail due to invalid location from where clang-format-diff.py is downloaded. Update the path to point to a stable, archived location. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7944 Test Plan: manually check the result of Github action. Reviewed By: ltamasi Differential Revision: D26345066 Pulled By: riversand963 fbshipit-source-id: 2b1a58c2e59c2f1eb11202d321d2ea002cb0917e
2020-11-04Update clang-format-diff.py (#7609)Xie Yanbo
Summary: `llvm-mirror/clang` is archived. Get the `clang-format-diff.py` file from the active source. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7609 Reviewed By: ajkr Differential Revision: D24711608 Pulled By: pdillinger fbshipit-source-id: b115d8765ff23fbb8190290a170de21565daba84
2020-05-21Clean up some code related to file checksums (#6861)Peter Dillinger
Summary: * Add missing unit test for schema stability of FileChecksumGenCrc32c (previously was only comparing to itself) * A lot of clarifying comments * Add some assertions for preconditions * Rename WritableFileWriter::CalculateFileChecksum -> UpdateFileChecksum * Simplify FileChecksumGenCrc32c with shared functions * Implement EndianSwapValue to replace unused EndianTransform And incidentally since I had trouble with 'make check-format' GitHub action disagreeing with local run, * Output full diagnostic information when 'make check-format' fails in CI Pull Request resolved: https://github.com/facebook/rocksdb/pull/6861 Test Plan: new unit test passes before & after other changes Reviewed By: zhichao-cao Differential Revision: D21667115 Pulled By: pdillinger fbshipit-source-id: 6a99970f87605aa024fa540c78cd519ff322c3e6
2020-04-30Add Github Action for some basic sanity test of PR (#6761)Yanqin Jin
Summary: Add Github Action to perform some basic sanity check for PR, inclding the following. 1) Buck TARGETS file. On the one hand, The TARGETS file is used for internal buck, and we do not manually update it. On the other hand, we need to run the buckifier scripts to update TARGETS whenever new files are added, etc. With this Github Action, we make sure that every PR does not forget this step. The GH Action uses a Makefile target called check-buck-targets. Users can manually run `make check-buck-targets` on local machine. 2) Code format We use clang-format-diff.py to format our code. The GH Action in this PR makes sure this step is not skipped. The checking script build_tools/format-diff.sh assumes that `clang-format-diff.py` is executable. On host running GH Action, it is difficult to download `clang-format-diff.py` and make it executable. Therefore, we modified build_tools/format-diff.sh to handle the case in which there is a non-executable clang-format-diff.py file in the top-level rocksdb repo directory. Test Plan (Github and devserver): Watch for Github Action result in the `Checks` tab. On dev server ``` make check-format make check-buck-targets make check ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6761 Test Plan: Watch for Github Action result in the `Checks` tab. Reviewed By: pdillinger Differential Revision: D21260209 Pulled By: riversand963 fbshipit-source-id: c646e2f37c6faf9f0614b68aa0efc818cff96787
generated by cgit v1.2.3-70-g09d2 (git 2.46.0) at 2024-10-22 22:28:05 +0000