[gentoo-dev] AUTHORS file for portage repository

[William Hubbs]

All,

based on the previous thread about copyright attribution clarifications,
I want to add the following AUTHORS file to the top level of the portage
repository if no one objects.

This is based on the description of the AUTHORS file at Google [1].

Everyone is not required to be listed, but there is no reason you can't
add yourself if you have contributed to the tree and want to be listed.

I hope this will satisfy everyone involved in the discussion.

Thoughts?

William

[1] https://opensource.google.com/docs/releasing/authors/

[Michał Górny]

Post by William Hubbs
All,
based on the previous thread about copyright attribution clarifications,
I want to add the following AUTHORS file to the top level of the portage
repository if no one objects.
This is based on the description of the AUTHORS file at Google [1].
Everyone is not required to be listed, but there is no reason you can't
add yourself if you have contributed to the tree and want to be listed.
I hope this will satisfy everyone involved in the discussion.
Thoughts?

Will that actually solve any problem, or just add another half-baked
solution with no benefit to the resulting status? In other words, would
SIE suddenly stop requiring you to alter copyright in ebuilds?

[Rich Freeman]

Post by Michał Górny
Will that actually solve any problem, or just add another half-baked
solution with no benefit to the resulting status? In other words, would
SIE suddenly stop requiring you to alter copyright in ebuilds?

++

It makes sense to ensure that the solution actually solves the problem
before we simply implement it.

If we really need such a file it would probably also make more sense
to have it auto-generated from git commit headers, which could use a
standardized format. This would actually provide more useful data
around authorship/copyright than a generic file with a list of names
anyway. Certainly standards like SPDX should be leveraged. These
tags could be optional in git, but anybody who wants to use these tags
could do so and tools that parse them could be created by those
interested in this information.

--
Rich

--
Rich

[Matt Turner]

Post by Rich Freeman
It makes sense to ensure that the solution actually solves the problem
before we simply implement it.
If we really need such a file it would probably also make more sense
to have it auto-generated from git commit headers

And how do you want to determine whether William's contributions are
copyright Sony or now? Do you want to look up his timezone and check
whether they were made during work hours?

If this satisfies Sony, please don't bikeshed this. The perfect it the
enemy of the good.

[Rich Freeman]

Post by Matt Turner

Post by Rich Freeman
It makes sense to ensure that the solution actually solves the problem
before we simply implement it.
If we really need such a file it would probably also make more sense
to have it auto-generated from git commit headers

And how do you want to determine whether William's contributions are
copyright Sony or now? Do you want to look up his timezone and check
whether they were made during work hours?

No, you look at the Copyright-owner header or whatever we want to call
it, and use that. Companies that care about labeling what they own
can take the time to properly document this.

[Mike Gilbert]

Post by Rich Freeman

Post by Matt Turner

Post by Rich Freeman
It makes sense to ensure that the solution actually solves the problem
before we simply implement it.
If we really need such a file it would probably also make more sense
to have it auto-generated from git commit headers

And how do you want to determine whether William's contributions are
copyright Sony or now? Do you want to look up his timezone and check
whether they were made during work hours?

No, you look at the Copyright-owner header or whatever we want to call
it, and use that. Companies that care about labeling what they own
can take the time to properly document this.

I would prefer not to see copyright noise to git commit messages.

If a manually maintained file will suffice, please don't
over-complicate matters.

[Matt Turner]

Post by Rich Freeman

Post by Matt Turner

Post by Rich Freeman
It makes sense to ensure that the solution actually solves the problem
before we simply implement it.
If we really need such a file it would probably also make more sense
to have it auto-generated from git commit headers

And how do you want to determine whether William's contributions are
copyright Sony or now? Do you want to look up his timezone and check
whether they were made during work hours?

No, you look at the Copyright-owner header or whatever we want to call
it, and use that. Companies that care about labeling what they own
can take the time to properly document this.

What Copyright-owner header are you talking about? You've been the
most outspoken opponent of using what appears to be the standard
attribution form specified in GLEP-76, and now that we have what I
think is a really good compromise you're against that too?

I know mailing list debates are your personal pastime but this is a
real wasteoftime.

[Rich Freeman]

Post by Matt Turner
What Copyright-owner header are you talking about?

We would create one, just as we've created bugzilla tags in git for
closing bugs/etc. Surely putting one line in a commit is no more
difficult than putting one file in a repository? Indeed anybody can
start sticking such a tag in commits today without any involvement
from anybody else.

Post by Matt Turner
You've been the
most outspoken opponent of using what appears to be the standard
attribution form specified in GLEP-76

When have I been opposed to anything in GLEP 76? I'll admit that I
don't 100% agree with everything in there, but I'm fine with following
the GLEP as it is written. Multi-line copyright notices aren't in
there, and the intent was never to be accumulating copyright holders
on the single notice line. An authors file was a compromise I wasn't
a huge fan of, but I'm suggesting that if we have one it ought to be
auto-generated (presumably with the work being done by somebody who
actually wants the file to be there).

Also, GLEP 76 as it is written says: "Projects using this scheme must
track authorship in a VCS, unless they list all authors of
copyrightable contributions in an AUTHORS file."

So, a VCS is the PREFERRED way of doing it. The alternative is
listing ALL authors in the authors file. Right now it seems like
people are advocating for only listing some authors.

Post by Matt Turner
I know mailing list debates are your personal pastime but this is a
real wasteoftime.

You're the one advocating for changing the status quo. I'm happy if
we drop the whole topic entirely. You certainly can't point fingers
at others when you're proposing doing something differently. We
wouldn't be having this discussion if some contributors were unwilling
to contribute under our current standards.

[Kent Fredric]

On Tue, 27 Nov 2018 15:10:36 -0500

Post by Rich Freeman
Also, GLEP 76 as it is written says: "Projects using this scheme must
track authorship in a VCS, unless they list all authors of
copyrightable contributions in an AUTHORS file."

Idea: How about using VCS as a defacto set of AUTHORS, but *also* support
an AUTHORS file that is designed to extend the content that git
provides generically.

That way you can just say something like:

"if the name appears naturally in git shortlog, you don't need to add
anything to the AUTHORS file"

And then git2rsync conversion can unify the two input sources.

nb: generating the AUTHORS file from git is naturally very time
consuming, as it requires full traversal of the entire repository.

However, there are practical ways of caching this (eg: generate it,
record the SHA1 it was generated at, then, next time, simply traverse
the subrange between HEAD and SHA1-last and update the cache based on
that).

But that very much puts this in the realm of "things that are painful
for end consumers to actually do themselves"

[Matt Turner]

Post by Rich Freeman

Post by Matt Turner
What Copyright-owner header are you talking about?

We would create one, just as we've created bugzilla tags in git for
closing bugs/etc. Surely putting one line in a commit is no more
difficult than putting one file in a repository? Indeed anybody can
start sticking such a tag in commits today without any involvement
from anybody else.

Post by Matt Turner
You've been the
most outspoken opponent of using what appears to be the standard
attribution form specified in GLEP-76

When have I been opposed to anything in GLEP 76? I'll admit that I

Now what I said. You've been the most outspoken opponent of using the
standard attribution format specified in GLEP-76. You know, the one
that says

Copyright YEARS MAIN-CONTRIBUTOR [OTHER-CONTRIBUTOR]... [and others]

and would suggest that it's allowable for Sony's name to be listed as
the MAIN-CONTRIBUTOR instead of Gentoo Authors.

Post by Rich Freeman
don't 100% agree with everything in there, but I'm fine with following
the GLEP as it is written. Multi-line copyright notices aren't in
there, and the intent was never to be accumulating copyright holders
on the single notice line. An authors file was a compromise I wasn't
a huge fan of, but I'm suggesting that if we have one it ought to be
auto-generated (presumably with the work being done by somebody who
actually wants the file to be there).
Also, GLEP 76 as it is written says: "Projects using this scheme must
track authorship in a VCS, unless they list all authors of
copyrightable contributions in an AUTHORS file."
So, a VCS is the PREFERRED way of doing it. The alternative is
listing ALL authors in the authors file. Right now it seems like
people are advocating for only listing some authors.

Let's not pretend that the authors of the GLEP (you're listed first,
by the way!) foresaw these issues (and rather obvious ones at that, I
might add). I'm already having to communicate the authors' intentions
and how they're different from what regular folks would think after
reading the GLEP (see:
https://github.com/gentoo/gentoo/pull/10481#issuecomment-442175181)

So let's satisfy everyone and be done with it: Let's put AUTHORS in
Git with a section header that states that these Copyright holders are
not obvious from the git history. This is where Sony would go. Then
let's append the output of "git log --format='%aN <%aE>" during
metadata generation or whenever stuff like that gets expanded. That
output is currently 36k FWIW.

Or, I don't know. Come up with something better and continue
bikeshedding. I won't.

Post by Rich Freeman

Post by Matt Turner
I know mailing list debates are your personal pastime but this is a
real wasteoftime.

You're the one advocating for changing the status quo. I'm happy if
we drop the whole topic entirely. You certainly can't point fingers
at others when you're proposing doing something differently. We
wouldn't be having this discussion if some contributors were unwilling
to contribute under our current standards.

At what point would you say maybe gentoo-{dev,proj}@ has heard enough
of me for a while? I'd wager that you have an order of magnitude more
emails to these lists this calendar year than commits to gentoo.git. I
see 20 commits and I'm not going to try count all your messages.

[Rich Freeman]

Post by Matt Turner
Or, I don't know. Come up with something better and continue
bikeshedding. I won't.

I think antarus already came up with something better - let Sony
explain its thinking, rather than trying to guess at what they're
trying to accomplish and whether it is something we want to support or
not.

Or just stick with what we've already been doing for the last 15
years, which is completely compatible with the new GLEP.

[Andrey Utkin]

Post by Matt Turner
So let's satisfy everyone and be done with it: Let's put AUTHORS in
Git with a section header that states that these Copyright holders are
not obvious from the git history. This is where Sony would go.

We can make it obvious from the git history, can't we?

commit ffffffffffffffffffffffffffffffffffffffff
Author: Dev on Payroll <***@bigcorp.com>
AuthorDate: <date>
Commit: Dev on Payroll <***@gentoo.org>
CommitDate: <date>

commit title

commit description

Signed-off-by: Dev on Payroll <***@bigcorp.com>
Signed-off-by: Dev on Payroll <***@gentoo.org>

[William Hubbs]

Post by Rich Freeman

Post by Michał Górny
Will that actually solve any problem, or just add another half-baked
solution with no benefit to the resulting status? In other words, would
SIE suddenly stop requiring you to alter copyright in ebuilds?

++
It makes sense to ensure that the solution actually solves the problem
before we simply implement it.
If we really need such a file it would probably also make more sense
to have it auto-generated from git commit headers, which could use a
standardized format. This would actually provide more useful data
around authorship/copyright than a generic file with a list of names
anyway. Certainly standards like SPDX should be leveraged. These
tags could be optional in git, but anybody who wants to use these tags
could do so and tools that parse them could be created by those
interested in this information.

My understanding is yes. SIE legal doesn't care where the copyright is
as long as it is there.

A good example of this file would be the one in the golang upstream
repo [1]. This file is not autogenerated and would only be updated as
requested when people or organizations want to add themselves.

I'm not sure auto generation is worth the work in this case since
there is very little overhead in maintaining the file. You just add a
person or organization when they request it. If they don't request it,
they don't care.

William

[1] https://raw.githubusercontent.com/golang/go/master/AUTHORS

[Andrey Utkin]

Post by William Hubbs
All,
based on the previous thread about copyright attribution clarifications,
I want to add the following AUTHORS file to the top level of the portage
repository if no one objects.
This is based on the description of the AUTHORS file at Google [1].
Everyone is not required to be listed, but there is no reason you can't
add yourself if you have contributed to the tree and want to be listed.
I hope this will satisfy everyone involved in the discussion.
Thoughts?

It seems to me this will grow huge, and be the source of annoyance for
users.

There's a plausible opinion that today's Unixes will stay around
forever:
https://utcc.utoronto.ca/~cks/space/blog/unix/DurableCurrentUnixes

And obviously Gentoo is the best flavour of them, so...

Are there any long-lived community FOSS projects maintaining such file?
FFmpeg had such one (CREDITS file), but eschewed it in 2013 for a short
notice "check git log".

[Matt Turner]

Post by Andrey Utkin

Post by William Hubbs
All,
based on the previous thread about copyright attribution clarifications,
I want to add the following AUTHORS file to the top level of the portage
repository if no one objects.
This is based on the description of the AUTHORS file at Google [1].
Everyone is not required to be listed, but there is no reason you can't
add yourself if you have contributed to the tree and want to be listed.
I hope this will satisfy everyone involved in the discussion.
Thoughts?

It seems to me this will grow huge, and be the source of annoyance for
users.

I'm imagining it's only those that are not obvious from the commit log
that will be listed here. But regardless, you're right that it will
grow. At the same time I doubt it will be a serious concern, and in
the case that it becomes one we can sort it out then.

[Ulrich Mueller]

Post by Andrey Utkin
It seems to me this will grow huge, and be the source of annoyance for
users.

IIUC the file has a specific purpose, namely to solve the copyright
attribution problem. So only those entities who would otherwise add
themselves to ebuild headers must be listed there.

Post by Andrey Utkin
There's a plausible opinion that today's Unixes will stay around
https://utcc.utoronto.ca/~cks/space/blog/unix/DurableCurrentUnixes
And obviously Gentoo is the best flavour of them, so...
Are there any long-lived community FOSS projects maintaining such file?

GNU Emacs:
http://git.savannah.gnu.org/cgit/emacs.git/tree/etc/AUTHORS
They add everyone who has contributed, and after 33 years the file
has grown to 170 kB, which I think is still acceptable. We have some
Manifest files that are much larger.

So I don't think we would run into problems anytime soon, even if we
added everybody (which we shouldn't, IMHO).

Ulrich

[William Hubbs]

Post by Ulrich Mueller

Post by Andrey Utkin
It seems to me this will grow huge, and be the source of annoyance for
users.

IIUC the file has a specific purpose, namely to solve the copyright
attribution problem. So only those entities who would otherwise add
themselves to ebuild headers must be listed there.

Post by Andrey Utkin
There's a plausible opinion that today's Unixes will stay around
https://utcc.utoronto.ca/~cks/space/blog/unix/DurableCurrentUnixes
And obviously Gentoo is the best flavour of them, so...
Are there any long-lived community FOSS projects maintaining such file?

http://git.savannah.gnu.org/cgit/emacs.git/tree/etc/AUTHORS
They add everyone who has contributed, and after 33 years the file
has grown to 170 kB, which I think is still acceptable. We have some
Manifest files that are much larger.
So I don't think we would run into problems anytime soon, even if we
added everybody (which we shouldn't, IMHO).

Agreed, we should not add every developer to this file by default.

On the other hand, if some person or organization requests to be added, we
should not refuse to add them, imho.

William

[Rich Freeman]

Post by William Hubbs
Agreed, we should not add every developer to this file by default.

Isn't this basically giving the most credit to the most
difficult-to-work-with entities by default?

As others have pointed out, it seems like other projects are moving
away from AUTHORS files in favor of git. If we want to go in the
opposite direction, why wouldn't we give credit to those who aren't
creating drama?

[Kent Fredric]

On Tue, 27 Nov 2018 12:51:00 -0500

Post by Rich Freeman
As others have pointed out, it seems like other projects are moving
away from AUTHORS files in favor of git.

"Other projects" don't typically have repos so large that a simple
application of a git filter-branch could take weeks.

That git manages not to die every day based on what we throw at it is
frankly a miracle of engineering.

[Rich Freeman]

Post by Kent Fredric
That git manages not to die every day based on what we throw at it is
frankly a miracle of engineering.

Our repo is a linked list being constantly manipulated from the head
backed by a hashed object store for the contents. For that use case
it is probably the ideal data structure. Since our use case is
actually the typical use case, it isn't a surprise that this was the
design that was chosen... :)

Computers are pretty fast when you actually use the correct algorithm...

[Kent Fredric]

On Tue, 27 Nov 2018 16:01:15 -0500

Post by Rich Freeman
Our repo is a linked list being constantly manipulated from the head
backed by a hashed object store for the contents. For that use case
it is probably the ideal data structure. Since our use case is
actually the typical use case, it isn't a surprise that this was the
design that was chosen... :)
Computers are pretty fast when you actually use the correct algorithm...

There's more to it than that. If that was all it was, then imagine if
it wasn't for all the compression and differencing tricks.

The raw size of an uncompressed verbatim, undifferential repository for
Gentoo would be phenomenal.

As it is, its fortunate we don't do a lot of things that *need* raw
access to non-tip commits, because doing so becomes very exhausting.

And were it not for its compression techniques and the fact our use of
Portage results in a vast number of highly-self-similar entries, then
we'd likely be slaughtered by disk IO alone, regardless of the linked
list approach.

Just don't try using filter branch on a whole gentoo repository, you'll
quickly learn why. ( You'll find yourself having to employ lots of
tricks with git fast-export instead just to avoid projected times in
weeks )

[Kent Fredric]

On Wed, 28 Nov 2018 10:11:32 +1300

Post by Kent Fredric
Just don't try using filter branch on a whole gentoo repository, you'll
quickly learn why. ( You'll find yourself having to employ lots of
tricks with git fast-export instead just to avoid projected times in
weeks )

Hah. Fun fact.

Right now, `git fast-export | wc -c` emits 1402067937

Which is 1337 Mb.

Such juxtaposition.

[Michał Górny]

Post by Rich Freeman

Post by Kent Fredric
That git manages not to die every day based on what we throw at it is
frankly a miracle of engineering.

Our repo is a linked list being constantly manipulated from the head
backed by a hashed object store for the contents. For that use case
it is probably the ideal data structure. Since our use case is
actually the typical use case, it isn't a surprise that this was the
design that was chosen... :)
Computers are pretty fast when you actually use the correct algorithm...

Yes, computers are fast and their work is cheap. On the other hand,
humans are not fast and their time is expensive. Now use the power of
human thinking to infer this to what you're doing to this thread.

[Rich Freeman]

Post by Michał Górny

Post by Rich Freeman

Post by Kent Fredric
That git manages not to die every day based on what we throw at it is
frankly a miracle of engineering.

Our repo is a linked list being constantly manipulated from the head
backed by a hashed object store for the contents. For that use case
it is probably the ideal data structure. Since our use case is
actually the typical use case, it isn't a surprise that this was the
design that was chosen... :)
Computers are pretty fast when you actually use the correct algorithm...

Yes, computers are fast and their work is cheap. On the other hand,
humans are not fast and their time is expensive. Now use the power of
human thinking to infer this to what you're doing to this thread.

Not wasting everybody's time with personal attacks?

[William Hubbs]

All,
I just picked a random msg to reply to on the thread.

Here is the updated AUTHORS file I would like to commit.

William

[Kristian Fiskerstrand]

Post by William Hubbs
All,
I just picked a random msg to reply to on the thread.
Here is the updated AUTHORS file I would like to commit.
William

Lets put it on agenda for next council meeting and let the discussion go
until then.

[Michał Górny]

Post by William Hubbs
All,
I just picked a random msg to reply to on the thread.
Here is the updated AUTHORS file I would like to commit.

You should include at least all current developers with commit access.
Because otherwise, this really looks like all Gentoo work was done by
the Foundation (which didn't do any work at all) and Sony, entirely
neglecting the huge effort done by many individuals.

I get you can't be expected to figure out all the people who ever done
any major contribution to Gentoo. However, that's no excuse to skip
the process entirely and just put your employer there.

[William Hubbs]

Post by Michał Górny

Post by William Hubbs
All,
I just picked a random msg to reply to on the thread.
Here is the updated AUTHORS file I would like to commit.

You should include at least all current developers with commit access.
Because otherwise, this really looks like all Gentoo work was done by
the Foundation (which didn't do any work at all) and Sony, entirely
neglecting the huge effort done by many individuals.

No, that is definitely not the case. all devs aren't required to be
listed; only those who want to be [1].

There is no way to contact everyone and see who wants to be listed, if
someone wants to be listed they can let us know.

William

[1] https://opensource.google.com/docs/releasing/authors/

[Craig Andrews]

Post by William Hubbs

Post by Michał Górny

Post by William Hubbs
All,
I just picked a random msg to reply to on the thread.
Here is the updated AUTHORS file I would like to commit.

You should include at least all current developers with commit access.
Because otherwise, this really looks like all Gentoo work was done by
the Foundation (which didn't do any work at all) and Sony, entirely
neglecting the huge effort done by many individuals.

No, that is definitely not the case. all devs aren't required to be
listed; only those who want to be [1].
There is no way to contact everyone and see who wants to be listed, if
someone wants to be listed they can let us know.
William
[1] https://opensource.google.com/docs/releasing/authors/

I think an AUTHORS file is very much a less than ideal solution.

To point out the absurdity, please include me - It'll be nice to see
only 3 names in the AUTHORS file with mine being first (hooray for
alphabetical order!)

Thanks,
~Craig

[Michał Górny]

Post by Craig Andrews

Post by William Hubbs

Post by Michał Górny

Post by William Hubbs
All,
I just picked a random msg to reply to on the thread.
Here is the updated AUTHORS file I would like to commit.

You should include at least all current developers with commit access.
Because otherwise, this really looks like all Gentoo work was done by
the Foundation (which didn't do any work at all) and Sony, entirely
neglecting the huge effort done by many individuals.

No, that is definitely not the case. all devs aren't required to be
listed; only those who want to be [1].
There is no way to contact everyone and see who wants to be listed, if
someone wants to be listed they can let us know.
William
[1] https://opensource.google.com/docs/releasing/authors/

I think an AUTHORS file is very much a less than ideal solution.
To point out the absurdity, please include me - It'll be nice to see
only 3 names in the AUTHORS file with mine being first (hooray for
alphabetical order!)

Please don't forget we're talking about Gentoo, so brace for 2 weeks of
bikeshedding over whether first or last name should come first.

[Rich Freeman]

Post by Michał Górny
Please don't forget we're talking about Gentoo, so brace for 2 weeks of
bikeshedding over whether first or last name should come first.

Thank goodness there isn't a Mike Gordon on the rolls or we could
discuss the intricacies of UTF-8 ordinality.

[Rich Freeman]

Post by William Hubbs

Post by Michał Górny

Post by William Hubbs
All,
I just picked a random msg to reply to on the thread.
Here is the updated AUTHORS file I would like to commit.

You should include at least all current developers with commit access.
Because otherwise, this really looks like all Gentoo work was done by
the Foundation (which didn't do any work at all) and Sony, entirely
neglecting the huge effort done by many individuals.

No, that is definitely not the case. all devs aren't required to be
listed; only those who want to be [1].

Uh, I'll see your [1] (a random non-Gentoo website) and raise you the
ACTUAL Gentoo policy on the matter [2].

If that policy is inappropriate you might want to revise it.

Post by William Hubbs
There is no way to contact everyone and see who wants to be listed, if
someone wants to be listed they can let us know.
[1] https://opensource.google.com/docs/releasing/authors/

[2] https://www.gentoo.org/glep/glep-0076.html#simplified-attribution

[Rich Freeman]

Post by Michał Górny

Post by William Hubbs
All,
I just picked a random msg to reply to on the thread.
Here is the updated AUTHORS file I would like to commit.

You should include at least all current developers with commit access.
Because otherwise, this really looks like all Gentoo work was done by
the Foundation (which didn't do any work at all) and Sony, entirely
neglecting the huge effort done by many individuals.
I get you can't be expected to figure out all the people who ever done
any major contribution to Gentoo. However, that's no excuse to skip
the process entirely and just put your employer there.

I did not realize that williamh is on Sony's payroll. For all the
talk of conflicts of interest I've seen regarding comrel decisions
(where there is no financial conflict of interest as just about every
company on the planet recognizes), I sincerely hope that williamh
intends to recuse himself from this matter as he has an obvious
financial conflict of interest here. Ditto for any other employees of
companies that wish to be acknowledged in our copyright notices.