BRIEF: The code round cannot really be a differentiator. It should be used as an elimination round and carry grades to next rounds.
REASON: Some people might want to do a bare minimum (because they are busy) and other who have more space can go overboard. It might be normalizing at your end, but you are ignoring that all test takers will not treat the test alike (its not a SAT for them).
DETAILS:
hmm what will a 4-8 hour coding challenge tell you that an open source project wont? If you give someone a coding challenge, possibly these are the things that you would rate the submission on. An open source project can equally show below.
1. Codeing test - Basic solution
a. Bare bone working application
b. Breaking complexities into testable parts
c. Usage documentation or a read me file
d. Commenting in code
Bonus points: for using any interesting algo
2. Architecture test - Improves over the basic solution (faster, more maintainable, etc)
a. Importing data
b. Database design
c. API design
d. Adding testcases
3. Product thinking - Improves over the problem statement
a. Building on top of the scope
4. Misc. (... stretching it a bit more)
a. Did I learn something new doing it? - tech wise, product wise, etc
b. How much time do I allocate for it?
c. When do I allocate time for it?
d. Is it similar to any older project, etc...?
Yes, I meant to agree with you that we can learn the same things from open source as a coding challenge, which is why we've considered it.
We're still a small company (but growing quickly). The code challenges are a simpler way to get the desired end goal. As we grow larger and have more capacity to devote to designing and refining an additional process around open source reviews, I think it's something we may explore. I'm not a decision-maker on that, so just my opinion. No disagreement in principle on this point, we just made a practical decision based on our constraints at the moment.
REASON: Some people might want to do a bare minimum (because they are busy) and other who have more space can go overboard. It might be normalizing at your end, but you are ignoring that all test takers will not treat the test alike (its not a SAT for them).
DETAILS:
hmm what will a 4-8 hour coding challenge tell you that an open source project wont? If you give someone a coding challenge, possibly these are the things that you would rate the submission on. An open source project can equally show below.
1. Codeing test - Basic solution
Bonus points: for using any interesting algo2. Architecture test - Improves over the basic solution (faster, more maintainable, etc)
3. Product thinking - Improves over the problem statement 4. Misc. (... stretching it a bit more) ((I had these listed somewhere in the mail))