Version Control for VHDL Project

[Mike Treseler]

You can only make that
statement if there are either no externals or every external is explicitly
revision specified.

And that is why I keep all referenced code
in the same repository and use no externals.
For me, externals are not worth all the brain cycles.
svn isn't perfect, but it works real good for free.

I agree.

Me too

Apologies to those clv readers that found this boring and off topic.

I think the discussion is on-topic as
lack of proper version control is a significant
problem in vhdl projects with multiple developers.

-- Mike Treseler

 

[Andy Peters]

If you really believe that then you should be able to explain why in my May
20 posting, I got different results from exporting the tag after modifying
only the trunk.  The answer, quite simply, (and I knew this going in and you
probably do too) is that the different results from exporting an unmodified
tag are a result of the lack of a revision being specified...OK, so what
mechanism exists to lock down revisions at the time of tag creation?
svncopy.pl is the only mechanism I've run across, but it has problems as
well.

So you are saying that you:

a) Tagged a submodule,
b) Added the submodule tag to the main project's source directory,
c) Did an svn update on the main project's source directory,
d) Committed the change to the main project's source directory.
e) Later checked out or exported the main project and discovered that
the submodule had the wrong stuff in it?

OK, I went back and found the 19 May post that describes your issue:

1. Setup a new repository
2. Add new folders called 'Project1' and 'Project2' to the repo.
3. Export 'Project1' to some folder
4. Add the svn_external property to the working copy of 'Project1' to reference 'Project2'.

By "reference 'Project2'" here, I assume that you mean you are
referencing Project2's TRUNK and not a TAG?

5. Do an update to get Project2 into the working copy.
6. Create a new text file called 'Project2_File.txt' in the Project2 folder.
7. Edit Project2_File.txt so it has the single line of text 'This is line #1'
8. Add Project2_File.txt into 'Project2' and commit the changes to both 'Project1' and 'Project2'.

At this point, Project2's __trunk__ now has the file
Project2_File.txt. You point out that it is likely that you will be
simultaneously working on both Project1 and its submodule Project2,
and so the external pointing to Project2's __trunk__ is correct.

9. Create your tag of 'Project1' using 'svn copy' or TortoiseSVN so that you
get the warm fuzzy feeling that you've successfully archived away the
current state of 'Project1' at this moment in time confident that you can
come back at any time to recover this entire project exactly as it was left
after step #8. To do so you will of course be specifying the exact revision
of 'Project1' to use to create the tag. Lets say this tagged version of
'Project1' is put into the archive at 'Tags/Project1_1.0'

********************************
And here is the problem:
You never created a tag for Project2.
********************************

The svn:externals property on Project1 probably looks like

Project2 svn://myrepo/Project2

So even though you created a tag for Project1, that tag still refers
to the head of Project2, and as such, if someone later modifies
Project2, you get those modifications, even though you checked out the
tagged version of Project1.

Project2 is its own project within the repo. So you will have to
create tags for it.

If you create a tag for Project2 (for example, Project2-v1.0), and
change the svn:externals property on Project1 to look like:

Project2 svn://myrepo/Project2/tags/Project2-v1.0/

then I __guarantee__ that this will work the way you want it to work.

---------------------------------------------
Assume that Project1 has two files, foo.txt and bar.txt, as well as
requiring the stuff in Project2. To ensure that we always get the
correct version of Project2, we've set the svn:externals property on
Project1 as noted above. Now we tag Project1 as Project1-v1.0. That
tag, which is a copy of Project1, also has the svn:externals property
referencing the tag Project2-v1.0.

So say we wish to check out the the Project1-v1.0 tag into the
directory /devel/Project1. Your working copy will look like:

/devel/Project1/ <- top of working copy
/devel/Project1/foo.txt <- a file in Project1
/devel/Project1/bar.txt <- another file in
Project1
/devel/Project1/Project2/ <- brought in by externals
/devel/Project1/Project2/Project2_File.txt <- the file in Project2

--------------------------------------------------------------------------

So your repo should be structured essentially like:

//myrepo/Project1/trunk <- working trunk for Project1
//myrepo/Project1/tags <- tags here
//myrepo/Project1/tags/Project1-v1.0 <- tag for 1st release
//myrepo/Project1/tags/Project1-v2.0 <- tag for 2nd release
//myrepo/Project2/trunk <- working trunk for Project2
//myrepo/Project2/tags <- tags here
//myrepo/Project2/tags/Project2-v1.0 <- tag for 1st release of
Project2
//myrepo/Project2/tags/Project2-v2.0 <- tag for 2nd release of
Project2

Does this help???

-a

 

[Marcus Harnisch]

So even though you created a tag for Project1, that tag still refers
to the head of Project2, and as such, if someone later modifies
Project2, you get those modifications, even though you checked out the
tagged version of Project1.

Project2 is its own project within the repo. So you will have to
create tags for it.

If you create a tag for Project2 (for example, Project2-v1.0), and
change the svn:externals property on Project1 to look like:

Project2 svn://myrepo/Project2/tags/Project2-v1.0/

then I __guarantee__ that this will work the way you want it to
work.

No. Because that is not the way KJ wants it to work as far as I
understand it. While this is not my development style, I think this is
a perfectly valid request.

1. KJ wants to be able to check out Project1, which refers
Project2. Whenever Project1 is checked out, the most recent
revision of Project2 should be checked out into the subdirectory of
Project1.

2. While he is working on Project1, he occasionally updates the
Project2 subdirectory to stay up-to-date. At some point KJ wants to
take a snapshot of his Project1 workspace *including* the current
state of all externals (i.e. Project2). In order to do that,
Subversion requires you to put explicit revision numbers into
externals property, which means that upon the next check out of
Project1, that same revision of Project2 is checked ut as well. But
that's not what he wants (see 1.). He doesn't want to be bothered
with revision numbers just to create a (recreatable) snapshot of
the current state of his workspace.

To (sort of) support KJ's development style, Subversion gives you two
basic options:

*Don't* fix the externals revision number in main-line. Whenever you
are about to create a snapshot, you'd create a branch of Project1,
switch your workspace over to that branch, add the current revision
number of Project2 to the externals property and commit again. Create
a tag for that revision.

*Do* fix the externals revision number in main-line. Whenever you
check out Project1, you will get the same revision of Project2 that
was current last time Project1 was checked in[1]. In order to track
the latest releases of externals, go into each sub directory and do
'svn update'. Take note of the new revisions and keep the externals
property of Project1 in sync. This could easily be wrapped into a
script, I reckon.

Regards
Marcus

Footnotes:
[1] Or rather: last time you *didn't forget* to update the externals
property.

 

[Charles Gardiner]

No. Because that is not the way KJ wants it to work as far as I
understand it. While this is not my development style, I think this is
a perfectly valid request.

1. KJ wants to be able to check out Project1, which refers
Project2. Whenever Project1 is checked out, the most recent
revision of Project2 should be checked out into the subdirectory of
Project1.

2. While he is working on Project1, he occasionally updates the
Project2 subdirectory to stay up-to-date. At some point KJ wants to
take a snapshot of his Project1 workspace *including* the current
state of all externals (i.e. Project2). In order to do that,
Subversion requires you to put explicit revision numbers into
externals property, which means that upon the next check out of
Project1, that same revision of Project2 is checked ut as well. But
that's not what he wants (see 1.). He doesn't want to be bothered
with revision numbers just to create a (recreatable) snapshot of
the current state of his workspace.

Hi,

from a work-flow view point, I see at least one major problem with this
methodology. If you are designing everything yourself in the course of
personal multi-tasking then it's fine, but as soon as a team is involved
I regard it as highly risky. The head should really carry the tag 'under
development'. I know several companies where the head is also used as a
sort of 'light-weight back-up'. A check-in is done on fridays, before
the developer goes on a course, holidays etc. and of course in larger
companies whenever the administrator needs to migrate the archive to a
different server (even if this is a seldom occurence). Also, it's very
useful to be able to check into the head from a workstation and pick it
up again from a lap-top when on the road and continue working.

Of course you could argue that the developer should make a branch first
and work there, merging back to trunk when he is finished. But this is
only forcing the responsibility on the developer of the reusable. The
flow is basically the same. You still need two 'streams' trunk/tags or
branches/trunk. I prefer the trunk/tags because it usually makes the
developer think of a tag name when he does the 'svn copy' even if it is
only 20090529a. But even that is better than simply 'r8080' or whatever.

An 'svn copy' to a tags or releases directory could be regarded as a
'contract'. The developer should be implicitly saying he has
fixed/improved/extended something, tested it and now makes it generally
available. IMHO, only reusables carrying this stamp should ever be
incorporated into a super-project.

Charles

 

[KJ]

Hi,

from a work-flow view point, I see at least one major problem with this
methodology. If you are designing everything yourself in the course of
personal multi-tasking then it's fine, but as soon as a team is involved
I regard it as highly risky.

Whether it's risky would depend on several other "usage-of-the-tool" factors
(more later), but I can say our software group has multiple people all
working concurrently and this is not an issue. However, they're also using
SourceSafe which then dictates a different type of work flow anyway, but the
bottom line is that when the integrator/builder person needs to create a
build they simply add a label to the top level project like
"xyz_Ver_1.0.10", then they check out that labelled version out and do their
build. I'm not trying to get into the relative merits/demerits of
SourceSafe versus Subversion here, my only points are:
- It is simple for the builder to create a new version that can then be
built
- There are no manual steps or checks that must be done to create the
labelled version, so there are no chances for mistakes when the those steps
get skipped. During crunch time, near product release, when time is very
valuable and scarce, the value of this probably should not be
underestimated.
- The build and test is a validation of what is in the trunk at a particular
moment. [1]
- That build can be recreated at any future time, regardless of what has
happened in the repository

The head should really carry the tag 'under
development'.

This seems to be an area where we differ somewhat. While I agree it is the
'under development' area, I don't think it should be used as scratch pad,
commits to the trunk should be understood to be contributing to the
development and should not 'break the build' or cause a test failure.

Also, it's very
useful to be able to check into the head from a workstation and pick it
up again from a lap-top when on the road and continue working.

That would be a misuse of commits in my opinion. It's just as easy to
create a patch file and put the patch on the laptop, no commits are
necessary to handle this situation.

Of course you could argue that the developer should make a branch first
and work there, merging back to trunk when he is finished.

That would be the other way to do it. For the 'pick it up on the laptop
later' scenario that you just described, the branch/merge would probably not
be a very good way for the developer to handle the situation, that's one
thing that patches are good for. For an extended development branch/merge
makes more sense. The point of creating the branch in the first place is to
totally decouple the widget from the trunk for the time being. It creates a
private work area for the developer...for now.

But this is
only forcing the responsibility on the developer of the reusable.

Since the developer is making the changes, this is where the responsibility
belongs.

Shoving the responsibility to each integrator to discern when the changes
are completed (what notifies them to do this) and then make possibly
numerous changes (i.e. to update all of his references to any newly changed
things) and putting this task on the critical path to the build (since the
task of updating those references can not even be started until after all
the developers creates their new tags) at a time of high stress (getting the
new build to fix the last problem/problems) is potentially error prone.

An 'svn copy' to a tags or releases directory could be regarded as a
'contract'. The developer should be implicitly saying he has
fixed/improved/extended something, tested it and now makes it generally
available. IMHO, only reusables carrying this stamp should ever be
incorporated into a super-project.

Contracts involve agreement between at least two parties and in this
instance (the creation of a new tag by a developer) there appears to be only
one, the developer. The creation of the new tag by a developer is not
anything the integrator would willingly agree to (but it is something that
can of course be imposed on them by local custom on how the version control
tool should be used). What it means to the integrator is that they must now
go in and update any references to the old thing that the developer made and
have it now point to the new thing. Multiply this task by having several
developers working on their own widgets and the integrator being responsible
for producing several different product builds and you quickly have a
potential mess...and ticked off integrator(s) who will greet each change as
an onerous burden rather than a relatively straight forward thing to handle.

Instead if the contract between integrator and developers is that things are
only committed to the trunk when they are "fixed/improved/extended" then the
integrator's work is straight forward, create a snapshot of the trunk (i.e.
create a release tag) and build it. Doing so has the following benefits:
- You haven't burdened the integrator to make changes in their stuff simply
to accomodate a new commit by a developer.
- You lessened the burden on the developer, there is no need for them to
create a new tag to follow the commit that they just did.
- You've lessened the need for problematic communication by the developer
announcing the availability of their new widget since now the developer does
not need to keep track of who all the integrators that might choose to use
their widget.
- The trunk then is the one place where one needs to go in order to get the
absolute latest/greatest/believed-to-be-best

Now of course the usual objection to this approach is that if the trunk is
the 'golden code' and a developer commits something that has some problem
with it then they've created a problem that needs to be fixed. OK, but the
problem exists, it is checked in, it needs to be fixed. With the approach
that you subscribe to, the developer has announced his new widget, the
integrator(s) cheerfully updates all of their references to the new widget,
does the builds and low and behold you're in the same spot, there is a
problem that needs fixing.

The only way you avoid having the problem appear is because of a breakdown
in communication between developer and integrator where the integrator was
unaware of the new widget and didn't update their references. Counting on a
breakdown in your system is probably not a 'good' thing to rely on. Finding
problems in the trunk early is better than finding them later. At crunch
time, if the problem can not be easily fixed, the integrator can choose to
roll back to the previous version by updating the references to widget to
point to the earlier working version. The solution and effort for this work
around is the same whether you follow my approach or yours, in my case
though this effort is the exception rather than standard operating
procedure.

Thanks again for explaining how you go about doing things, it's always good
to hear about different ways of approaching the same basic problem.

Kevin Jennings

[1] The assumption heres are that
- Trunk code that references reusable components does so via references to
components that do NOT specify specific revision levels (this would have to
be manually checked by the integrator, but also can be automated somewhat
and does not need to be a critical path thing to do)
- The creation of the tag or labelled version locks down specific revision
levels for that tag, but not to references that may be the trunk