Browse Source

rm -rf Github

danger
Duke Leto 4 years ago
parent
commit
bab65bf2a3
  1. 70
      contrib/devtools/README.md
  2. 58
      contrib/devtools/fix-copyright-headers.py
  3. 74
      contrib/devtools/git-subtree-check.sh
  4. 181
      contrib/devtools/github-merge.sh

70
contrib/devtools/README.md

@ -2,55 +2,11 @@
This directory contains tools for developers working on this repository.
github-merge.sh
==================
## security-check.py
A small script to automate merging pull-requests securely and sign them with GPG.
Perform basic ELF security checks on a series of executables.
For example:
./github-merge.sh bitcoin/bitcoin 3077
(in any git repository) will help you merge pull request #3077 for the
bitcoin/bitcoin repository.
What it does:
* Fetch master and the pull request.
* Locally construct a merge commit.
* Show the diff that merge results in.
* Ask you to verify the resulting source tree (so you can do a make
check or whatever).
* Ask you whether to GPG sign the merge commit.
* Ask you whether to push the result upstream.
This means that there are no potential race conditions (where a
pullreq gets updated while you're reviewing it, but before you click
merge), and when using GPG signatures, that even a compromised github
couldn't mess with the sources.
Setup
---------
Configuring the github-merge tool for the bitcoin repository is done in the following way:
git config githubmerge.repository bitcoin/bitcoin
git config githubmerge.testcmd "make -j4 check" (adapt to whatever you want to use for testing)
git config --global user.signingkey mykeyid (if you want to GPG sign)
fix-copyright-headers.py
===========================
Every year newly updated files need to have its copyright headers updated to reflect the current year.
If you run this script from src/ it will automatically update the year on the copyright header for all
.cpp and .h files if these have a git commit from the current year.
For example a file changed in 2014 (with 2014 being the current year):
```// Copyright (c) 2009-2013 The Bitcoin Core developers```
would be changed to:
```// Copyright (c) 2009-2014 The Bitcoin Core developers```
symbol-check.py
==================
## symbol-check.py
A script to check that the (Linux) executables produced by gitian only contain
allowed gcc, glibc and libstdc++ version symbols. This makes sure they are
@ -69,8 +25,7 @@ If there are 'unsupported' symbols, the return value will be 1 a list like this
.../64/test_bitcoin: symbol std::out_of_range::~out_of_range() from unsupported version GLIBCXX_3.4.15
.../64/test_bitcoin: symbol _ZNSt8__detail15_List_nod from unsupported version GLIBCXX_3.4.15
update-translations.py
=======================
## update-translations.py
Run this script from the root of the repository to update all translations from transifex.
It will do the following automatically:
@ -81,22 +36,7 @@ It will do the following automatically:
See doc/translation-process.md for more information.
gen-manpages.sh
===============
## gen-manpages.sh
A small script to automatically create manpages in ../../doc/man by running the release binaries with the -help option.
This requires help2man which can be found at: https://www.gnu.org/software/help2man/
git-subtree-check.sh
====================
Run this script from the root of the repository to verify that a subtree matches the contents of
the commit it claims to have been updated to.
To use, make sure that you have fetched the upstream repository branch in which the subtree is
maintained:
* for src/secp256k1: https://github.com/bitcoin/secp256k1.git (branch master)
* for sec/leveldb: https://github.com/bitcoin/leveldb.git (branch bitcoin-fork)
Usage: git-subtree-check.sh DIR COMMIT
COMMIT may be omitted, in which case HEAD is used.

58
contrib/devtools/fix-copyright-headers.py

@ -1,58 +0,0 @@
#!/usr/bin/env python
# Copyright (c) 2019-2020 The Hush developers
# Released under the GPLv3
from __future__ import print_function
'''
Run this script inside of src/ and it will look for all the files
that were changed this year that still have the last year in the
copyright headers, and it will fix the headers on that file using
a perl regex one liner.
For example: if it finds something like this and we're in 2014
// Copyright (c) 2009-2013 The Bitcoin Core developers
it will change it to
// Copyright (c) 2009-2014 The Bitcoin Core developers
It will do this for all the files in the folder and its children.
Author: @gubatron
'''
import os
import time
year = time.gmtime()[0]
last_year = year - 1
command = "perl -pi -e 's/%s The Hush/%s The Hush/' %s"
listFilesCommand = "find . | grep %s"
extensions = [".cpp",".h"]
def getLastGitModifiedDate(filePath):
gitGetLastCommitDateCommand = "git log " + filePath +" | grep Date | head -n 1"
p = os.popen(gitGetLastCommitDateCommand)
result = ""
for l in p:
result = l
break
result = result.replace("\n","")
return result
n=1
for extension in extensions:
foundFiles = os.popen(listFilesCommand % extension)
for filePath in foundFiles:
filePath = filePath[1:-1]
if filePath.endswith(extension):
filePath = os.getcwd() + filePath
modifiedTime = getLastGitModifiedDate(filePath)
if len(modifiedTime) > 0 and str(year) in modifiedTime:
print(n, "Last Git Modified: ", modifiedTime, " - ", filePath)
os.popen(command % (last_year,year,filePath))
n = n + 1

74
contrib/devtools/git-subtree-check.sh

@ -1,74 +0,0 @@
#!/bin/sh
DIR="$1"
COMMIT="$2"
if [ -z "$COMMIT" ]; then
COMMIT=HEAD
fi
# Taken from git-subtree (Copyright (C) 2009 Avery Pennarun <apenwarr@gmail.com>)
find_latest_squash()
{
dir="$1"
sq=
main=
sub=
git log --grep="^git-subtree-dir: $dir/*\$" \
--pretty=format:'START %H%n%s%n%n%b%nEND%n' "$COMMIT" |
while read a b junk; do
case "$a" in
START) sq="$b" ;;
git-subtree-mainline:) main="$b" ;;
git-subtree-split:) sub="$b" ;;
END)
if [ -n "$sub" ]; then
if [ -n "$main" ]; then
# a rejoin commit?
# Pretend its sub was a squash.
sq="$sub"
fi
echo "$sq" "$sub"
break
fi
sq=
main=
sub=
;;
esac
done
}
latest_squash="$(find_latest_squash "$DIR")"
if [ -z "$latest_squash" ]; then
echo "ERROR: $DIR is not a subtree" >&2
exit 2
fi
set $latest_squash
old=$1
rev=$2
if [ "d$(git cat-file -t $rev 2>/dev/null)" != dcommit ]; then
echo "ERROR: subtree commit $rev unavailable. Fetch/update the subtree repository" >&2
exit 2
fi
tree_subtree=$(git show -s --format="%T" $rev)
echo "$DIR in $COMMIT was last updated to upstream commit $rev (tree $tree_subtree)"
tree_actual=$(git ls-tree -d "$COMMIT" "$DIR" | head -n 1)
if [ -z "$tree_actual" ]; then
echo "FAIL: subtree directory $DIR not found in $COMMIT" >&2
exit 1
fi
set $tree_actual
tree_actual_type=$2
tree_actual_tree=$3
echo "$DIR in $COMMIT currently refers to $tree_actual_type $tree_actual_tree"
if [ "d$tree_actual_type" != "dtree" ]; then
echo "FAIL: subtree directory $DIR is not a tree in $COMMIT" >&2
exit 1
fi
if [ "$tree_actual_tree" != "$tree_subtree" ]; then
git diff-tree $tree_actual_tree $tree_subtree >&2
echo "FAIL: subtree directory tree doesn't match subtree commit tree" >&2
exit 1
fi
echo "GOOD"

181
contrib/devtools/github-merge.sh

@ -1,181 +0,0 @@
#!/bin/bash
# This script will locally construct a merge commit for a pull request on a
# github repository, inspect it, sign it and optionally push it.
# The following temporary branches are created/overwritten and deleted:
# * pull/$PULL/base (the current master we're merging onto)
# * pull/$PULL/head (the current state of the remote pull request)
# * pull/$PULL/merge (github's merge)
# * pull/$PULL/local-merge (our merge)
# In case of a clean merge that is accepted by the user, the local branch with
# name $BRANCH is overwritten with the merged result, and optionally pushed.
REPO="$(git config --get githubmerge.repository)"
if [[ "d$REPO" == "d" ]]; then
echo "ERROR: No repository configured. Use this command to set:" >&2
echo "git config githubmerge.repository <owner>/<repo>" >&2
echo "In addition, you can set the following variables:" >&2
echo "- githubmerge.host (default git@github.com)" >&2
echo "- githubmerge.branch (default master)" >&2
echo "- githubmerge.testcmd (default none)" >&2
exit 1
fi
HOST="$(git config --get githubmerge.host)"
if [[ "d$HOST" == "d" ]]; then
HOST="git@github.com"
fi
BRANCH="$(git config --get githubmerge.branch)"
if [[ "d$BRANCH" == "d" ]]; then
BRANCH="master"
fi
TESTCMD="$(git config --get githubmerge.testcmd)"
PULL="$1"
if [[ "d$PULL" == "d" ]]; then
echo "Usage: $0 pullnumber [branch]" >&2
exit 2
fi
if [[ "d$2" != "d" ]]; then
BRANCH="$2"
fi
# Initialize source branches.
git checkout -q "$BRANCH"
if git fetch -q "$HOST":"$REPO" "+refs/pull/$PULL/*:refs/heads/pull/$PULL/*"; then
if ! git log -q -1 "refs/heads/pull/$PULL/head" >/dev/null 2>&1; then
echo "ERROR: Cannot find head of pull request #$PULL on $HOST:$REPO." >&2
exit 3
fi
if ! git log -q -1 "refs/heads/pull/$PULL/merge" >/dev/null 2>&1; then
echo "ERROR: Cannot find merge of pull request #$PULL on $HOST:$REPO." >&2
exit 3
fi
else
echo "ERROR: Cannot find pull request #$PULL on $HOST:$REPO." >&2
exit 3
fi
if git fetch -q "$HOST":"$REPO" +refs/heads/"$BRANCH":refs/heads/pull/"$PULL"/base; then
true
else
echo "ERROR: Cannot find branch $BRANCH on $HOST:$REPO." >&2
exit 3
fi
git checkout -q pull/"$PULL"/base
git branch -q -D pull/"$PULL"/local-merge 2>/dev/null
git checkout -q -b pull/"$PULL"/local-merge
TMPDIR="$(mktemp -d -t ghmXXXXX)"
function cleanup() {
git checkout -q "$BRANCH"
git branch -q -D pull/"$PULL"/head 2>/dev/null
git branch -q -D pull/"$PULL"/base 2>/dev/null
git branch -q -D pull/"$PULL"/merge 2>/dev/null
git branch -q -D pull/"$PULL"/local-merge 2>/dev/null
rm -rf "$TMPDIR"
}
# Create unsigned merge commit.
(
echo "Merge pull request #$PULL"
echo ""
git log --no-merges --topo-order --pretty='format:%h %s (%an)' pull/"$PULL"/base..pull/"$PULL"/head
)>"$TMPDIR/message"
if git merge -q --commit --no-edit --no-ff -m "$(<"$TMPDIR/message")" pull/"$PULL"/head; then
if [ "d$(git log --pretty='format:%s' -n 1)" != "dMerge pull request #$PULL" ]; then
echo "ERROR: Creating merge failed (already merged?)." >&2
cleanup
exit 4
fi
else
echo "ERROR: Cannot be merged cleanly." >&2
git merge --abort
cleanup
exit 4
fi
# Run test command if configured.
if [[ "d$TESTCMD" != "d" ]]; then
# Go up to the repository's root.
while [ ! -d .git ]; do cd ..; done
if ! $TESTCMD; then
echo "ERROR: Running $TESTCMD failed." >&2
cleanup
exit 5
fi
# Show the created merge.
git diff pull/"$PULL"/merge..pull/"$PULL"/local-merge >"$TMPDIR"/diff
git diff pull/"$PULL"/base..pull/"$PULL"/local-merge
if [[ "$(<"$TMPDIR"/diff)" != "" ]]; then
echo "WARNING: merge differs from github!" >&2
read -p "Type 'ignore' to continue. " -r >&2
if [[ "d$REPLY" =~ ^d[iI][gG][nN][oO][rR][eE]$ ]]; then
echo "Difference with github ignored." >&2
else
cleanup
exit 6
fi
fi
read -p "Press 'd' to accept the diff. " -n 1 -r >&2
echo
if [[ "d$REPLY" =~ ^d[dD]$ ]]; then
echo "Diff accepted." >&2
else
echo "ERROR: Diff rejected." >&2
cleanup
exit 6
fi
else
# Verify the result.
echo "Dropping you on a shell so you can try building/testing the merged source." >&2
echo "Run 'git diff HEAD~' to show the changes being merged." >&2
echo "Type 'exit' when done." >&2
if [[ -f /etc/debian_version ]]; then # Show pull number in prompt on Debian default prompt
export debian_chroot="$PULL"
fi
bash -i
read -p "Press 'm' to accept the merge. " -n 1 -r >&2
echo
if [[ "d$REPLY" =~ ^d[Mm]$ ]]; then
echo "Merge accepted." >&2
else
echo "ERROR: Merge rejected." >&2
cleanup
exit 7
fi
fi
# Sign the merge commit.
read -p "Press 's' to sign off on the merge. " -n 1 -r >&2
echo
if [[ "d$REPLY" =~ ^d[Ss]$ ]]; then
if [[ "$(git config --get user.signingkey)" == "" ]]; then
echo "ERROR: No GPG signing key set, not signing. Set one using:" >&2
echo "git config --global user.signingkey <key>" >&2
cleanup
exit 1
else
git commit -q --gpg-sign --amend --no-edit
fi
else
echo "Not signing off on merge, exiting."
cleanup
exit 1
fi
# Clean up temporary branches, and put the result in $BRANCH.
git checkout -q "$BRANCH"
git reset -q --hard pull/"$PULL"/local-merge
cleanup
# Push the result.
read -p "Type 'push' to push the result to $HOST:$REPO, branch $BRANCH. " -r >&2
if [[ "d$REPLY" =~ ^d[Pp][Uu][Ss][Hh]$ ]]; then
git push "$HOST":"$REPO" refs/heads/"$BRANCH"
fi
Loading…
Cancel
Save