Still Learning

How Open Source Projects Survive Poisonous People (And You Can Too)

54 min - Jan 25, 2007 Open Source project: Subversion

This video is by Ben and Brian from Subversion community currently working on Google Code from Chicago. This video has not a single second of technical content but focused fully on human side of an open source development process and their worst enemy - poisonous people. This is one of those few open discussions on social side of project development. I am sure it happens in corporate world too but we can’t identify or talk about them. Although they acknowledge at the end but most of the talk is relevant to any community or project group in corporate. So it will not hurt your time if you are not in OSS project. [I have also not contributed to any project till now! ]

Their discussion made me realize that how much each project has history, emotions and vision behind it. Today when you see thousands of projects in successful open source communities, we seem to take things for granted. Talk is distillation of their experience over 6++ yrs project(s), hence information presentation is concise and precise.

Complete presentation is broken down into following for major sections:

• Comprehend - preserve attention and focus
• Fortify - build a healthy community
• Identify - look for tell-take signs
• Disinfect - maintain calm and stand your ground.

Other discussed areas are:

Documentation:

One important thing pointed out by Ben and Brian was the value of documentation. Although they have stressed and pointed out its relevance in terms of OSS world, but I see how much we can learn from it. Incremental documentation can turn into a good book, or precise instructions are useful in providing URL to newcomer etc. What I want to learn is how open source developer is able to maintain and write documentation along with software. Since I see that even in corporate world it fluctuates between two extremes — too much or none at all!

Key things to track as part of project history are:

• design decisions
• bug fixes
• Mistakes
• change log / code change

On Bus factor:

Bus factor defines how many people it takes to be hit by bus (job change, health issues, marriage etc. etc.) to take project off rail. This is very critical for open source project, hence always focus on increasing your bus-factor. I can say that this is also very critical for commercial software teams.

Some other tidbits of advice:

• Send commit emails, encourages email review.
• Do not submit big changes
• Don’t out author name at the top as it marks the territory. Let it give the feel of community.

I happened to see an old video on channel 9 where Loius and Ale are discussing exception handling in details. This is a real informative as they discuss how exception handling is done in Win32 and 64 bit, why it is done so, cost of exception handling and usage, exception across DLLs and modules.

Ale Contenti and Louis Lafreniere: Understanding Exceptions and When/How to Handle Them
Video link: http://channel9.msdn.com/showpost.aspx?postid=343189

During the video I captured some notes and putting it here without any formatting. So look this to get basic idea but spend an hr to listen to them.

They mention blog post by Chris Brumme on exception handling which is _the_ informative dissection on the subject. Post is here.

Notes:

Exceptions generated in C++, machine level (SEH) and generated by code via throw. Performance impact varies a lot depending on whether user wants to catch all exceptions or only program generated (throw).

5.00 - Should catch everything at the top level. Catch and then die. C++ runtime actually catches all (SEH and throw) the exceptions. User should not put catch (…) in main but let it pass to CRT and let it handle.

7.30 Anatomy of exception. Two pass handling of an exception.

void f()
{
throw std::outof.range()
}

void g()
{
try
{

}
except(0)
{

}
}

void h()
{
try
{
g()
}
catch (const std::outof.range& e)
{
}
}

int main()
{
h()
}

stack:
f -> search handler, nothing so go down
g -> except(0) - filter
h
main

13.00 (time)
Two pass:
1. Look for handler..
2. If handled then go back and call destructor

if no handler then CRT will call terminate.

compiler swithc /EHs - does not run any destructors and dies immediately.

24.40[time]
Performance in exception handling.

Win32: How exception handling is implemented (SEH level)

void f()
{
Obj obj;
throw std::outof.range();
}

How OS keep track of exceptions:: using linked list at FS:0 keeps track of all frames which has exception. Need to link and unlink pointers at FS:0 everything we enter f(). Since f() has created obj, it needs to be destroyed. Keep track of state from -1 up and down based on obj creation and destructor (this has some overhead). If we can figure out which obj don’t need exception handling and then don’t keep track of state of that object.

33:30[time]
What is the right thing to do?

40time]: Current compiler framework is mainly C and new Phoneix compiler is in C++ with very minimal exception throwing - when compiles throws up hands.

47[time]: Throwing exception across DLL is not a good idea. It is tricky and harder to handel.

50[time]: Destructor should never throw. If during exception handling (unwinding) if destructor throws then program terminates.

So, I finally made to my first BanCamp and overall experience was really encouraging. Financial sponsors had made sure that we got plentiful of coffee, fruits and pizza. Thanks a lot for feeding the crowd.
I spend half a day at MinneBar sessions and talking to few people. I enjoyed time spend tremendously as it was informative on many levels. Most disappointing thing to me was that most of the people were from web design and development community. I talked with Ben and may be next year I will try to have a session on product development technologies (C/C++) and associated ecosystem.

First session that I attended was on “Small Teams, Big Results” by Ben Edwards .
This was very relevant to me as I worked for 4 yrs in a large company and team and last years working in 3 people team. Ben talked about challenges for large company like size, meetings, documentation, agility etc. For me even small teams have issues with product marketing, communication, estimation etc. Still very active discussion as lot of people irrespective of their work or team background could corelate easily. (Slides might come up at Refacr site here, not available at the time of post)

After this I went to panel discussion on “Design Coding”, which was organized by Ben Edwards (Refactr), Norm Orstad (Slantwise), Mike Bollinger (Livefront), Margaret Andrews (Vita.mn), Darrell Austin (Darrel Austin Designs).
Talk content was not relevant to me but it was amusing to see that designers think that developers are from other world and don’t understand them, also through out 1 hr session no one talked about end user experience or how much designers are responsible for it. Something else which surprised me was that no one talked about new technologies like Silverlight, Flex, etc. but were concerned with that they need to learn Javascript and most of their learning is self-taught. For a designer it seems that now a days you need to learn a lot on technology side apart from school learning to be able to work in Industry. I see many parallels with Comp Sci and regular software development work. Learning keeps on going….

Final panel seesion that I attended was “State of the State: Technology in Minnesota - Panel Discussion” with panelists having lot of diverse and varied experience. Panelists included Douglas Olson (Microsoft), Jamie Thinglestad (Dow Jones), Michael Gorman (Split Rock Partners), Robert Stephens (Geek Squad), Dan Grigsby (Unpossible), Matthew Dornquast (code42).
This was very interactive and energetic panel. Topics discussed were from the how to start company here, whether to move it to West or not and why Minnesotans can’t create environment like Silicon Valley. Matthew D. gave some realistic advice based on his experience in developing CrashPlan and gave one interesting metaphor - Silicon Prairie, his vision to see MN transform into Silicon Valley equivalent. Douglas O. shared his involvement in bringing MS here and his other prior ventures. Gorman shared VC’s point of view and iterated that geographic locations has not much to do with the success of the company. He encouraged developers to bring in marketing and finance people on board, if they ever want to turn brigh and fantastic idea into successful one. This discussion was worth my time! Most important take away shared by the panel was “Minnesotans are risk averse, so just go out and take risk. Do it early and fail fast”.

This is my first experience and hope that next time I can do lot more.

Tomorrow morning, I am planning to spend half a day at MinneBar being organized at U of M. To see the sessions check out here

As of now I am planning to take part in following sessions:

• Small Teams, Big Results
• Metaprogramming in Groovy
• Rough Spots with Agile Development
• Distributed Teamwork: How to succeed with a team you never see
• Stackless Python Twisted-Style — There will be blood

Also if slot is open then I plan to have a session ” Do you C++?” — I want to meet with other developers mainly in product development area to understand and explore what language they are using and what are the challenges.

I am sure most of us have started using shared_ptrs when Boost made it available to the world and if you have not then I would suggest start doing it. I came across this excellent post by Stephan (STL); developer on VC++ libraries about his detailed presentation on TR1::shared_ptr — which is essentially boost::shared_ptr but developed by Dinkumware.

If you want to learn the shared_ptr usage then go through an impressive slide deck (48 slides) only on shared_ptr here.

Slides have lots of intro-level material while covering most of the usage scenarios; so it is normal to ask why to talk about such basic stuff? Here is Stephan’s comment on it, which I agree fully. As many C++ developers are at intermediate level rather than guru!

“Not all professional C++ programmers have the opportunity to use Boost at work, nor do all of them choose to spend their free time, as I do, using Boost at home. Additionally, these slides can be useful even to Boost-experienced programmers. For example:

1. Our shared_ptr allocator support is a C++0x feature, backported to VC9 TR1 (and Boost 1.35, which isn’t out yet).
2. Understanding shared_ptr’s internal representation is useful to get a feel for how it behaves.
3. It’s important to know where Boost differs from TR1. Boost.Regex’s grammar contains stuff that TR1 doesn’t (for example \< and \>). Also, Boost.Regex supports some function calls that TR1 doesn’t (see regex slide 28; I filed a Library Issue about that).
4. Boost’s documentation is very good, but even users of Boost.Regex might not be familiar with regex_iterator and regex_token_iterator; my slides provide a useful overview of what they do.

Even the STL is unfamiliar to some professional C++ programmers; the best way to deal with this is education.
Stephan T. Lavavej, Visual C++ Libraries Developer“

Pointer to member function seems to be less known concept in C++, but I happen to came indirectly while reading about ScopeGuard by Andrei via GotW #88.
Idea is that instead of passing pointer to function, we can use pointer to member function of specific class. When I started writing, I came across is beautiful and well written article by Michael D. Crawford titled “Pointers to C++ Member Functions“. Please read it in detail to understand different usage or read code below to figure out.

// Usage of pointer to member function syntax.

#include

class Obj {
int _id;
public:
Obj(int id): _id(id) {}
int getX2() { return _id * 2; }
int getX4() { return _id * 4; }
int getX8() { return _id * 8; }
};

typedef int (Obj::*GetXX)(); // Declare pointer-to-member-func

int main()
{
Obj t1(10);
GetXX get = &Obj::getX8; // Assign member function to the pointer-to-member

  int val = (t1.*get)();
std::cout << " val is " << val << std::endl; // Invoke getX8() returns 80
}

There is vast amount of documentation and articles covering usage, pitfalls and design of exception handling in C++ code. So instead of writing another new article, I will focus on what I have learned and what others have preached.

I have been coding in C++ for 2 yrs now and at work, we tend to avoid using exceptions deep in engine but use it at appropriate boundaries. We are still biased against error code as preferred way of getting error state from deep inside. I tend to use exception based on following practices when possible:

• Use exception at boundaries and critical places.
• Avoid using inside any tight for loops or functions (avoid your top 3-4 costly functions)
• Use possible fall back instead of propagating exception where possible, e.g using defaults when file path is invalid, fall back to default algorithm in absence of required extensions etc.

There are many more valuable articles written by experts on this topic, so I will try to cite and discuss major ones, which I have referenced or read over time.

Exceptional Design by Hubert Matthews at ACCU - Few remarks made in the article are very apt and worth iterating here. To enjoy it, read original article.

• Placement of catches is determined by recovery point based on business requirements
• Many throws, few catches
• Catches at module boundaries - e.g. in main() [OS<->Program] etc
• Implement strong exception safety guarantee when reasonable.
• Guideline: View exceptions as recovery requests, move from the notion of “stuff happens” to “what am I going to do about it” thinking.
• Why do programmers fails?
• Habit and old idioms
• low-level detail
• suspicion of exceptions
• Few exception anti-patterns
• try/catch surrounding each func call
• Exception squashing
• Overuse of exceptions
• throws in catch blocks
• NULL recovery block - try {.. } catch (const std::exception& e) {}

Other references:

Exception Safety in Generic Components by David Abrahams on Boost.org

Abrahams exception safety guarantees

– Basic - no memory leak, state can change
– Strong - transactional commit/rollback - state does not change,
– nothrow - no failed operations

Using ScopedGuard by Andrei and Petru: Contains excellent workflow of how we throw exceptions out of code apart from design and implementation details of ScopedGuard

Scott Collins: Detailed article and references to various exception usage - I Love C++ Exceptions

Rico’s articles