Wednesday, April 8, 2015

More on ThreadRAII and Thread Suspension

On March 31, I did a webcast for O'Reilly on material from Effective Modern C++. (The webcast was recorded, and the recording is available here.) The focus of the presentation was how void futures can be used to communicate between threads in situations where condition variables and/or boolean flags might otherwise be used. The example I employed was starting a thread in a suspended state, and the code I ultimately built to was this:
{
  std::promise<void> p;                        // created first,
                                               // destroyed last
  std::thread t2([&p]
                 {
                   try {
                     p.get_future().get();
                     funcToRun();
                   }
                   catch(...) { … }
                 }
                );             

  ThreadRAII tr(std::move(t2),                  // created after p,
                ThreadRAII::DtorAction::join);  // destroyed before p

  …

  p.set_value();

  …

}                                   // if p hasn’t been set,tr’s
                                    // dtor hangs on a join
I explained that this code has the drawback that if control flows through this block such that the ThreadRAII object tr is created, but p.set_value() isn't executed (e.g., because an exception gets thrown), tr will hang in its destructor waiting to join with a thread that will never finish. For details, watch the webcast or consult pages 266-269 of the printed book (i.e., the second half of Item 39).

In Effective Modern C++, I conclude my discussion of this matter with "There are ways to address this problem, but I’ll leave them in the form of the hallowed exercise for the reader," and I refer to a blog post I made in December 2013 that, along with the comments it engendered, examines the issue in detail. In the webcast, I offer an explanation for why I don't show how to deal with the problem: "I don't know of a solution that's clean, simple, straightforward, [and] non-error-prone."

Michael Lindner took me to task for this. He wrote, "I was disappointed at the end of the webcast on void futures, because there was hype about void futures being useful, but the only example was a non-working one with no solution." Ouch. He went on to suggest that it may only be necessary to join with t2 if it's unsuspended, i.e., if funcToRun is permitted to run. If we don't need to synchronize with the completion of funcToRun, Michael suggested, we could do a detach instead of a join. This would mean that before the call to set_value, tr's DtorAction should be detach, and it should be set to join only after set_value has been called. Adding a setter to ThreadRAII for its DtorAction would make that possible, and the code could look like this:
{
  std::promise<void> p;
                       
  std::thread t2([fut = std::move(p.get_future())]
                 {
                   try {
                     fut.get();
                     funcToRun();
                   }
                   catch(...) { … }
                 }
                );

  ThreadRAII tr(std::move(t2),                      // before set_value is called,
                ThreadRAII::DtorAction::detach);    // configure tr's dtor to detach

  …

  p.set_value();
  tr.setDtorAction(ThreadRAII::DtorAction::join);   // after set_value is called,
                                                    // configure tr's dtor to join

  …

}                               // if p hasn't been set, its dtor sets an exception,
                                // thus preventing tr's dtor from hanging
With this approach, if set_value isn't called, p's destructor will write a std::future_error exception into the shared state that the future accesses, and that will cause the call to get in t2 to unblock. t2 will thus be able to run to completion. We won't know when t2 will complete, and we certainly can't guarantee that it will occur before the block containing tr finishes executing, but this will avoid the problem in my code whereby tr's destructor blocks waiting for a call to set_value that never occurs.

An interesting subtlety of this code is the lambda's capture clause, which performs what amounts to a move capture of the future produced by p. This is in contrast to the reference capture of p present in my code. My code performs a join on t2, and t2's lifetime is less than p's, so p must exist while t2 runs. It's therefore safe for t2 to refer to p. In Michael's approach, the detach means it's possible for t2 to continue running after p has been destroyed. It's thus critical that t2 not access p, and the move capture of p's future arranges for that to be the case. I'm grateful to Tomasz Kamiński for bringing this issue to my attention, though I'll note that Yehezkel Bernat suggested the same thing (presumably on stylistic grounds) in group chat during the webcast.

But what if we really need to do an unconditional join with t2--even if funcToRun is never permitted to execute? In that case, a detach-based approach won't work. The fact that std::promise's destructor writes an exception into its shared state seems like it should be able to be used as the basis of a solution, but there appears to be a chicken-and-egg problem:
  • In order to avoid having tr hang in its destructor, p must be destroyed before tr. 
  • That means that p must be defined after tr, because objects are destroyed in the inverse order of their construction.
  • tr must be initialized with the function its thread will execute, and that function must wait on a future that we provide.
  • To get a future, we need a std::promise.
  • That means that p must be defined before tr, because we need p to produce the future for initialization of tr.
  • Bullets 2 and 5 are contradictory.
Tomasz Kamiński sent me some code that showed how to avoid the apparent contradiction. The essence of his approach is to define tr before p, but to initialize it without a function to run on the std::thread it wraps. Without a function to run, there is no need for a future on which that function should block. After p has been defined, tr can receive an assignment that specifies the function to run and the action to perform in tr's destructor. This function can block on a future in the usual manner. Tomasz uses a class to encapsulate the various subtleties in his solution (e.g., tr must be declared before p, tr must be default-initialized and then assigned to). In this code (which is based on his implementation, but is not identical to it), I'm assuming that ThreadRAII has been revised to support default construction and that a default-constructed ThreadRAII object contains a default-constructed std::thread, i.e., a std::thread with no function to execute:
class SuspendedTask {
public:
  // FuncType should be callable as if of type void(std::future<void>)
  template<typename FuncType> 
  explicit SuspendedTask(FuncType&& f)
  {                         // no member init list ==> default-construct tr and p

    tr = ThreadRAII(std::thread([fut = p.get_future(), f = std::forward<FuncType>(f)]{ f(std::move(fut)); }), 
                    ThreadRAII::DtorAction::join); 
  }
 
  std::promise<void>& get_promise() { return p; }
 
private:
  ThreadRAII tr;            // tr must be listed 
  std::promise<void> p;     // before p!
};
This class can be used as follows:
{
  SuspendedTask t2([](std::future<void> fut){
                     try {
                       fut.get();
                       funcToRun();
                     }
                     catch(...) { … }
                   });

  …

  t2.get_promise().set_value();

  …

}
A key element of this design is a class that bundles together a std::thread (inside the ThreadRAII object) and a std::promise. That idea was the crux of the first comment on my December 2013 blog post; it was made by Rein Halbersma. Other commenters on that post embraced the same idea, but as the thread went on, things got increasingly complicated, and by the end, I wasn't really happy with any of the solutions presented there. That's why I didn't use any of them in the book. In retrospect, I should have spent more time on this problem.

I'm grateful to Michael Lindner and Tomasz Kamiński for pushing me to revisit the problem of thread suspension through void futures, ThreadRAII, and exception-safe code.

Friday, April 3, 2015

Materials Update for my C++11/14 and my C++-in-Embedded Training Courses

Artima just pushed out the latest updates to my annotated training materials on C++11/14 and on making effective use of C++ in an embedded environment. These materials were originally published in 2010, and in the ensuing five years, I've updated the C++11/14 materials nine times and the C++-in-embedded materials twice--a total of ten and three releases, respectively.

From the beginning, the rule has been that buyers are entitled to free updates to the materials as long as I produce them, so if you own a copy of either (or both) set/sets of materials, you should have received email notification from Artima that shiny, new PDF is waiting for you in your Artima account.

I don't plan to update these materials further. C++11 and 14 have been finalized, and I think by the time C++ is updated again (a process that, practically speaking, is likely to occur both through technical specifications as well as a revised language standard), training courses covering the latest features will assume that people have a background in much of C++11 and C++14. As regards C++ in embedded systems, my current materials focus on C++98/03, because my experience has been that embedded shops tend to be more conservative in their adoption of new compilers (and the language features they offer) than their non-embedded counterparts. Going forward, I think it would be prudent for training courses for embedded developers to assume that C++11 features are (or soon will be) available, and I don't plan to revise my training course to adopt that stance. (That's because my current C++-related work builds on my experience with and the information in Effective Modern C++.)

I think the just-published versions of these training courses correspond well to the practical state of C++ as of 2015. If you've purchased copies of these materials, I hope you find the latest updates useful. (You should have received a copy of the changelog in the email telling you about the availability of the new versions.) If you don't have copies of these materials, I think they're a good way to educate yourself about the topics they address. As always, you can download free samples of each set of materials from the following links:
Scott



Friday, March 20, 2015

EMC++ News: New Excerpt, Upcoming Webcast, 50% Ebook Discount

There's lots going on in the Effective Modern C++ 'verse these days. Behind the scenes, I'm gearing up to  revise the book for an upcoming forth printing, during which I hope to make a big dent in the list of known errata, but in the meantime, please note the following:
  • The Effective Modern C++ Sample Page has been updated to include a new sample Item. This time it's

    Item 14: Declare functions noexcept if they won't emit exceptions.

    As with the previous sample Items, this is the final version of preliminary material I published on my blog for community feedback, most recently about a year ago.
  • On March 31, I'll be doing a live webcast covering material from the book's chapter on the C++11 concurrency API. The Item I'll be officially discussing is Item 39, "Consider void futures for one-shot event communication," but the presentation will include information from other Items, as well. For details on the webcast and to sign up (it's free), mosey on over to the webcast's official page.
  • Through March 25, O'Reilly is running a C++ promotion that makes digital versions of its C++ titles available for 50% off. To take advantage of this opportunity to cut my royalties in half, follow this link.
Scott

Thursday, March 19, 2015

New ESDS Book: Effective Python

Brett Slatkin's Effective Python is the latest book with a full-color interior to join my Effective Software Development Series, and I'm very pleased about it.The title of the first chapter--Pythonic Thinking--establishes the approach of the book, and Brett's clear writing style drives it home.

But why take my word for it? The book's web site currently has the following sample Items available for your reading and learning pleasure:

Item 40: Consider Coroutines to Run Many Functions Concurrently
Item 23: Accept Functions for Simple Interfaces Instead of Classes
Item 34: Register Class Existence with Metaclasses
Item 53: Use Virtual Environments for Isolated and Reproducible Dependencies
Item 17: Be Defensive When Iterating Over Arguments

I hope you enjoy Effective Python.

Scott

Saturday, February 21, 2015

Why auto deduces std::initializer_list for a braced initializer

For some years, I've wondered aloud why
auto x { 1, 2, 3 };
deduces x's type as std::initializer_list, even though the braced initializer itself ("{1, 2, 3}") has no type. One example of my out-loud wondering took place on this blog about a year ago ("If braced initializers have no type, why is the committee so insistent on deducing one for them?"). Another was during my talk at CppCon 2014, "Type deduction and why you care". James Hopkin recently watched the video of that presentation, and he then sent me the following quite interesting message:
You ask at one point for any info on the motivation for N3922's special case for auto. I did a bit of digging and thought I'd send you what I found.

The short story is that N2640 proposed the special case that auto should deduce braced initializers as initializer_lists, without realizing that doing so broke uniform initialization (e.g. it makes int x{7}; and auto x{7}; very different). N3922 fixes that by (of course!) introducing another special case: single parameter braced initializers have their own rule.

Slightly more detail: N2640 tries to keep template argument deduction simple but attempts to allow a braced initializer to be passed to two or more functions via assigning it to an auto. This got turned into wording in N2672. Note that Stroustrup's previous design in N2532 allows deduction of initializer_lists for both unconstrained template parameters and auto, which is more consistent but also breaks uniform initialization.

None of this explains why N3922 didn't just remove the special case for auto. That wouldn't have resulted in silent changes to the meaning of code and would have simplified the language.
For purposes of understanding why there's a special rule for auto type deduction and braced initializers, the key sentence in N2640 is this:
On the other hand, being able to deduce an initializer_list for T is attractive to
allow:
auto x = { 1, 1, 2, 3, 5 };
f(x);
g(x);
which was deemed desirable behavior since the very beginning of the EWG discussions about initializer lists.
That's not a lot of detail, but it's more than I've ever seen before. Thank you, James Hopkin, for digging up this background information!

Scott

Monday, February 16, 2015

New EMC++ Sample Content is Now Online

O'Reilly has updated the Effective Modern C++ Sample Page to include a new book excerpt. This time it's

Item 7: Distinguish between () and {} when creating objects.

This is the published version of an Item I originally posted in draft form in a blog post about 11 months ago.

Enjoy!

Scott

Monday, February 2, 2015

Expressions can have Reference Type


Today I got email about some information in Effective Modern C++. The email included the statement, "An expression never has reference type." This is easily shown to be incorrect, but people assert it to me often enough that I'm writing this blog entry so that I can refer people to it in the future.

Section 5/5 of the Standard is quite clear (I've put the relevant text in bold):
If an expression initially has the type “reference to T” (8.3.2, 8.5.3), the type is adjusted to T prior to any further analysis. The expression designates the object or function denoted by the reference, and  the expression is an lvalue or an xvalue, depending on the expression.
There'd clearly be no need for this part of the Standard if expressions couldn't have reference type.

If that's not enough to settle the matter, consider the type of an expression that consists of a function call. For example:
        int& f();                // f returns int&
        auto x = f();            // a call to f
        
What is the type of the expression "f()", i.e., the type of the expression consisting of a call to f? It's hard to imagine anybody arguing that it's not int&, i.e., a reference type. But what does the Standard say? Per 5.2.2/3 (where I've again put the relevant text in bold and where I'm grateful to Marcel Wid for correcting the error I had in an earlier version of this post that referred to 5.2.2/10):
If the postfix-expression designates a destructor (12.4), the type of the function call expression is void; otherwise, the type of the function call expression is the return type of the statically chosen function (i.e., ignoring the virtual keyword), even if the type of the function actually called is different. This return type shall be an object type, a reference type or cv void.
It's very clear that expressions can have reference type. Section 5/5 takes those expressions and strips the reference-ness off of them before doing anything else, but that's not the same as the reference-ness never being present in the first place.

Scott

Friday, January 30, 2015

EMC++ Sample Content

O'Reilly has decided it's time to turn up the heat on the grand Effective Modern C++ marketing campaign. A key component of that campaign is giving away excerpts of the book, the idea being that if you like the stuff we give away, you're more likely to plunk down some cash for the rest of the book.

We're starting with Chapter 1 ("Deducing Types"), which consists of these Items:
Item 1: Understand template type deduction.
Item 2: Understand auto type deduction.
Item 3: Understand decltype.
Item 4: Know how to view deduced types.
To download this free sample, visit the freshly-minted landing page that O'Reilly has set up for it.

This material has actually been available at the book's web site for some time, but there, it's available only as PDF. Now we're making it available in all the digital formats in which the book is published: PDF, ePub, and Mobi. This means you can see how the book will look on your digital device of choice. On some devices, there are some display issues (dutifully chronicled as part of the book's errata list), but we're working on those, and we remain committed to producing a book that looks as good as possible on all platforms we support. As we resolve problems, we update the digital files and make them available to everybody who has purchased them.

We plan to release additional book excerpts according to the following schedule:
  • Week of February 9: Item 7: Distinguish between () and {} when creating objects.
  • Week of February 23: Item 14: Declare functions noexcept if they won’t emit exceptions.
  • Week of March 9: Item 42: Consider emplacement instead of insertion.
Sometime in March, we're also planning to have me do a live webcast covering some of the material in the book. The date and the material are yet undecided. If you'd like to participate in the decision process, feel free to let me know what Item(s) you'd like to hear me walk you through during the webcast. I will consider your suggestions carefully before doing whatever I want :-)

Enjoy the sample material from Effective Modern C++!

Scott

PS - Currently, the O'Reilly web site lists the print version of EMC++ as out of stock and due to be available in 2025. I'm guessing--hoping!--that the due date is a typo, but the book is in stock and available for your immediate purchase at Amazon and elsewhere.

50% off EMC++ eBook due to O'Reilly's "Best of January" Promotion

If you'll pardon my bragging a bit, Effective Modern C++ has recently nudged David Pogue's OS X Yosemite: The Missing Manual aside to retake the #1 position on O'Reilly's bestseller list. As such, it qualifies for O'Reilly's "Best of January" promotion, whereby they offer a discount code good for 50% off their 10 best-selling videos and 10 best-selling ebooks. The discount code in question is DEAL.

Despite the "One Day Only" bannerette on the promotion page, the discount runs through the weekend. Hey, one day, three days--who cares if you get 50% off?

Scott

Saturday, January 10, 2015

Nine New Videos: Three Legit, Six Bootleg

Videos of my November presentations at code::dive in Wrocław, Poland, and at ConFu in Jurmala, Latvia, have recently been posted for your post-holiday time-killing pleasure.


The titles of my code::dive talks are "CPU Caches and Why You Care" and "Support for Embedded Programming in C++11 and C++14". To understand the comments I make at the beginning of the latter talk, it will be helpful to view Venkat Subramaniam's talk on Lightweight Design, because his talk--which is worth viewing both for its merit and for the fact that Venkat is such an engaging speaker--immediately preceded mine, and it motivates my opening remarks that might otherwise make no sense, including my decision to give my talk without shoes.

At ConFu, I was scheduled to give two talks, a keynote on interface design and a full-day tutorial on material in Effective Modern C++ (EMC++). The slot for my keynote was at the end of the day, but at the conference's opening keynote, I noticed that the room setup made it nearly impossible for the people in the back of the room to see the bottom third of the screen. I therefore spent the next several hours reformatting my slides to avoid using more than about the top two thirds of the screen, and I was very pleased with the way the talk went. The topic is "The Most Important Design Guideline," and though I've written and spoken about this guideline before (the guideline is to make interfaces easy to use correctly and hard to use incorrectly), this talk is the first time I felt like I really found a way to say what I wanted to say. If you're not convinced about the importance of the guideline from this presentation, I'm not going to be able to convince you.

The ConFu organizers and I had agreed that my EMC++-based tutorial wasn't going to be recorded, but somebody attending the tutorial decided to do it on his or her own. The audio and video quality is what you'd expect from an informal recording, but since it's at YouTube, I figure you might as well know about it. It's in six videos of about an hour each (which makes sense, because I give tutorial attendees a break every hour), and you'll find links to all six at my Online Videos Page. In fact, that page links to videos of each of my presentations and interviews that I'm aware of. If you're aware of an online video that should be included there, but isn't, please let me know.

Happy viewing!

Scott

Wednesday, December 17, 2014

The Evolving Search for Effective C++

The video of my keynote address at Meeting C++ 2014 on December 5 has just been posted to the Meeting C++ Channel at YouTube. I was given a long time slot (two hours), so I addressed two rather different topics, both based on my work of the past quarter century identifying and promulgating guidelines for effective C++ programming.

The meat of the first topic is an explanation of how one of the guidelines in Effective Modern C++--"Consider emplacement instead of insertion"--came to have the form that it does. It has a lot of hard-core C++ content, though I hope that the secondary lesson about the importance of seeking feedback on prospective programming guidance also comes through. This part of the talk runs about an hour.

The second topic is about disseminating information in written and presentation form, and it's not about C++ at all. Rather, it's about issues I think authors, publishers, and trainers need to consider when packaging technical information for consumption in an age where writing is in print or digital form and where the reach of a technical talk is often much greater in recorded form than through a live presentation. This part of the video is about 30 minutes long.

At the end of the talk is some 20 minutes of Q&A.

I worked hard preparing this presentation, and I'm pleased with the way it came out and with the material I covered. I hope you enjoy it.

Scott

Friday, December 5, 2014

Effective Modern C++ Exists!

I arrived in Berlin last night (where I'm participating in Meeting C++) to find a package waiting for me with two print copies of EMC++! At about the same time, I found out that the initial print run had sold out by the time it hit the warehouse, so a second printing has been scheduled in about a week. If you didn't get a copy of the book from the initial print run, don't feel bad: the books I ordered for some training I'll be doing next week got pushed back to the second printing, too. The only copies I'll see for a while are the two you see in the picture.

For my Meeting C++ keynote this morning, O'Reilly gave me a discount code for both print and digital versions of EMC++, and since I shared it with the conference attendees, I'll share it with you, too:  AUTHD. That should save you a sweet 50% off the ebook and a nearly-as-sweet 40% off the print book. (Inexplicably, the discount isn't applicable to the print+ebook bundle, so if you want both, put them into your cart individually, because the total price will then beat the already-nicely-discounted bundle pricing.)

In the meantime, Addison-Wesley has extended their deep ebook Deal of the Day discount on the digital versions of my earlier C++ books to a second day, so if you're wondering what to do while you're waiting for your print copy of EMC++ to arrive, don't overlook the possibility of clicking and shopping for EC++, MEC++, and ESTL.

Scott


Thursday, December 4, 2014

Special Sale: Digital Bundle of EC++, MEC++, and ESTL for $40!

Addison-Wesley has chosen my Effective C++ Digital Collection (i.e., digital versions of Effective C++, Third Edition, More Effective C++, and Effective STL) as their December 4 ebook Deal of the Day. The bundle, which normally sells for $95.99, has been marked down to $39.99. At only about $13 per book, I think that's a very attractive price.

These books have not been updated for C++11, but most of the information in them remains fundamentally sound, and of course if you haven't yet migrated to C++11, they're as useful now as they've ever been. (For my perspective on the relevance of the information in Effective C++ in the C++11 era, take a look at this blog post.)

Happy shopping!

Scott




Friday, November 28, 2014

First Review of Effective Modern C++


The first review (that I'm aware of) of the "published" version of EMC++ just appeared at the CoderGears Blog. The review gives these five reasons to read the book:
  1. It explains well the motivations behind the standards.
  2. It shows the relation between the new features.
  3. Provides some basic and pragmatic rules to modernize your C++ code base.
  4. Samples are very easy to understand.
  5. Smart pointers demystified.
The review concludes:
In the final this book is really Effective, it explains the motivations behind the new features and help you  understand the whole picture about the new standards. Beginer or even expert C++ developers will find it very useful.
Check out the full review for all the details.

For addition comments on the book, take a look at the reviews at O'Reilly's web site. Currently all the comments there refer to pre-publication drafts that we released, but I'm pleased to report that everybody gave those drafts five stars out of five.

Speaking of the O'Reilly web site, they're currently running the de rigueur post-Thanksgiving sale, so ebooks (including EMC++) are 50% off until December 2. That means you can pick up the digital version of EMC++ for $21.49 via the apparently-inspired-by-FORTRAN66 discount code CYBERDY. You can order here.

Scott

Wednesday, November 19, 2014

EMC++ Display Issues on Kindle

I've received some reports about how Effective Modern C++ doesn't render well on Kindle, with code comments not aligning correctly and a narrow single-character ellipses ("…") being visually indistinguishable from a wider three-character ellipses ("..."). In response to my queries about these kinds of problems, my production team at O'Reilly told me:
One of the Kindle options available is "Publisher Font." If the user selects this font, the comments will be aligned and the difference between the ellipses will be visible. None of the other font options available on the Kindle device will display the alignment in the proper way. By default, Kindle devices don't default to Publisher Font, which is a shame. Users will have to make the selection manually.
I've noted some other Kindle-related issues at the EMC++ Errata List. If you find additional problems, let me know, and I'll see what we can do about addressing them in revised versions of the digital versions of the book. (Revised versions are free to people who've bought the digital version of EMC++.)

Scott

Monday, November 10, 2014

EMC++ Exits Publishing Purgatory!

I just received the following from O'Reilly:

Congratulations! Your book went to print on Friday, and we've now completed production. The Retail Availability Date for ebooks on oreilly.com and Amazon is tomorrow, 11/11. For print books, it's estimated at 12/2. 
Finally!

Scott

Monday, October 20, 2014

Wanted: Good & Bad UI Examples from Mobile Realm

From time to time, I give a presentation on what I consider to be the most important guideline in software development: Make interfaces easy to use correctly and hard to use incorrectly. Over the years, I've collected a variety of interface examples that either adhere to this rule or, more commonly, violate it, and I try to update the presentation to keep the examples contemporary.

I'd like to add some examples from the mobile realm (smart phones, ebook readers, tablets), but--gasp!--I don't have any such devices. Yes, I'm stuck in a technological time warp hovering in the 1990s. But you're not, and I'm hoping you'll be willing to share with me examples of mobile interfaces that are easy to use correctly and hard to use incorrectly...or that aren't. If you could provide screen shots, that'd be ideal.

I break "easy to use correctly and hard to use incorrectly" down into the following sub-guidelines, so interfaces that either uphold these ideas or trample all over them would be especially helpful:
  • Adhere to the principle of least astonishment.
  • Choose good names.
  • Be consistent.
  • Use progressive disclosure.
When I talk about interfaces, that includes APIs, so I'd be happy to receives examples of good and bad API designs for mobile platforms, too.

Thanks for any help you can offer.

Scott

Sunday, October 19, 2014

New ESDS Book: Effective Ruby

I'm pleased to welcome another book with a full-color interior into my Effective Software Development Series, Peter J. Jones' Effective Ruby.

I'm not a Ruby developer, but Peter's writing is a pleasure to read, and his experience with and enthusiasm for the language is clear. For those of you who do code in Ruby, I encourage you to check out this very nice book.

Scott

Monday, September 22, 2014

The Four Stages of Doneness

People have been asking me when Effective Modern C++ will be done. (My wife is among them, and our dog has been giving me funny looks, too.) For my earlier books, the "when will you be done" question was easy to answer, because my deliverable was camera-ready PDF. A book was done when I sent in the final PDF.

For EMC++, I've decided that it makes sense to break "done" down into four stages:
  • Done, Stage 1, is when I send in a "final" manuscript for typesetting.
  • Done, Stage 2, is when I submit information to be used for preparing the index.
  • Done, Stage 3, is when I approve the typeset versions of the book.
  • The final stage of doneness is when I approve the index. At that point, the book is ready for publication.
We're currently in the middle of Stage 3. O'Reilly send me their initial cut of PDF and digital files (i.e., epub and Kindle-compatible mobi) for the manuscript I submitted at Stage 1. The process of my revising these files is known as QC1. I've sent back my QC1 comments, and I expect to get QC2 files in a few days. QC1 didn't include the index, but the QC1 index is supposed to show up in my inbox any time now.

In the meantime, work on the book's cover is moving along, and above you can see what I expect to be the final front cover image. We're also honing in on the text to go on the back of the book. O'Reilly's marketing department and I have gone through four iterations so far. It's not easy to describe in ~250 words what a ~300-page technical book is about and why you absolutely positively must buy it, but we're doing our best :-).

I continue to expect the book to be available in October.

I'll keep you posted on the book's progress. Nobody wants this book to be fully done more than I do, but it's important that it be not just done, but done well. The final two stages of doneness will help ensure that that's the case. When it comes out, it will be the first book I've written that was designed to render well on both paper and digital devices. If you consume it digitally, it should look as good (and be as useful) regardless of whether you prefer portrait or landscape, color or monochome, and large or small fonts. The only formatting assumption I make is that I can fit 64 monospaced characters on a line, so some code examples could get a bit whacked out on small cell phones, but other than that, my goal is that EMC++ should look good and convey useful information no matter how you read it.

Scott

Tuesday, September 9, 2014

CppCon Hair Poll


During my talk at CppCon yesterday, I decided to liven things up by asking the audience to answer a question that comes up often: who does my hair style make me look like? People volunteer answers to this question--which, until now, I've never asked--on a surprisingly frequent basis.
I wasn't able to show the results of the poll in real time, because I didn't realize that if I got more than 100 responses, I'd have to pay to see the data. Serves me right for forgetting the fine print I'm sure I agreed to when I started with SurveyMonkey.

I've now ransomed the data, and the final results (when CppCon ended) are as follows:







Wednesday, September 3, 2014

50% Off Effective Modern C++ Ebook

From now through September 9, O'Reilly's running a sale: all EBooks are 50% off, including the Early Release version of Effective Modern C++.  (Buying the early release version also gets you a copy of the final digital edition when it comes out, which is slated to be in about a month.)

My book is currently at the top of the O'Reilly Best Seller List (okay, it's not so much at the top as in the upper left corner, but I choose to interpret that in the best possible light), so if you've been thinking of buying the digital version of the book, this is a great way to help keep EMC++ #1 while making sure that O'Reilly and I don't make much money off of it  :-)


Scott

Monday, August 25, 2014

Upcoming Appearances: 2 Continents, 5 Countries, 8 Cities, Several Topics

Now that work on Effective Modern C++ is approaching completion, it's time to get out and see the world...and give technical presentations. Between September and December, I'll be giving talks in the United States, Germany, England, Poland, and Latvia. Geographically, the European presentations work out this way:
In the USA, I'll be in Bellevue, WA, Cambridge, MA, and Bala Cynwyd, PA.

The topics I'll be addressing are rather varied, including a lot of information about C++11 and C++14, the importance of CPU caches for software performance, general advice on interface design, plus some reflections on thinking about and writing "Effective" books.

You'll find details on all my upcoming presentations at my Upcoming Talks page, e.g., exact dates, locations, and topics. I think all the presentations will be fun, but let me call out two in particular:
  • If you're interested in material from Effective Modern C++, the most comprehensive seminar I'll be giving on that topic will take place on October 7-8 in London.
  • A special attraction of C++ and Beyond in Stuttgart is looking like it might be a full-blown bar brawl between me and Herb Sutter. In recent weeks, he and I have been going back and forth about the wisdom (or lack thereof) of advising people to use pass-by-value in function interfaces. We've exchanged some posts on this topic in comments on my blog and in posts to a Microsoft C++ MVP mailing list, plus there have been some behind-the-scenes email messages, and at this point, having carefully weighed all the facts, it looks like the only thing we fully agree on is that the other person means well, but is terribly misguided. Herb's German may be better than mine (it definitely is), but when it comes to parameter-passing advice, that boy is going down!
I hope to see you in Bellevue, Cambridge (MA), Bala Cynwyd, Stuttgart, London, Wrocław, Riga, or Berlin later this year.

Scott

Wednesday, August 20, 2014

Near-Final Draft of Effective Modern C++ Now Available (plus TOC and sample Item)

Effective Modern C++ is moving closer and closer to reality. This post contains:
  • Information about availability of an almost-final draft of the book.
  • The current (and probably final) table of contents.
  • A link to the I-hope-I-got-it-right-this-time version of my Item on noexcept.

Draft Book Availability

A revised and nearly-final manuscript is now available through O'Reilly's Early Release Program and Safari Books Online's Rough Cuts program. The prose still needs to be sanded down a bit, but practically speaking, this is the final stream of words that will make up the book. Remaining tasks include double-checking the code samples, index generation and final formatting (i.e., typesetting), but this should be quite close to what gets published.

I've left the line numbers in for the online drafts, because that makes it easier for people to report errors to me. The really truly honest-this-is-it version of the manuscript is due to O'Reilly in early September, so if you see anything that needs improvement, please let me know by the end of this month!

Probably-Final Table of Contents

Here's the current table of contents. I'm not wild about the title of the final chapter ("Tweaks"), so if you have suggestions for a better title, let me know.
CHAPTER 1  Deducing Types
  Item 1:  Understand template type deduction. 
  Item 2:  Understand auto type deduction. 
  Item 3:  Understand decltype. 
  Item 4:  Know how to view deduced types. 

CHAPTER 2  auto
  Item 5:  Prefer auto to explicit type declarations. 
  Item 6:  Use the explicitly typed initializer idiom when auto deduces 
           undesired types. 

CHAPTER 3  Moving to Modern C++
  Item 7:  Distinguish between () and {} when creating objects. 
  Item 8:  Prefer nullptr to 0 and NULL. 
  Item 9:  Prefer alias declarations to typedefs. 
  Item 10: Prefer scoped enums to unscoped enums. 
  Item 11: Prefer deleted functions to private undefined ones. 
  Item 12: Declare overriding functions override. 
  Item 13: Prefer const_iterators to iterators. 
  Item 14: Declare functions noexcept if they won't emit exceptions. 
  Item 15: Use constexpr whenever possible. 
  Item 16: Make const member functions thread-safe. 
  Item 17: Understand special member function generation. 

CHAPTER 4  Smart Pointers
  Item 18: Use std::unique_ptr for exclusive-ownership resource management. 
  Item 19: Use std::shared_ptr for shared-ownership resource management. 
  Item 20: Use std::weak_ptr for std::shared_ptr-like pointers that can dangle.
  Item 21: Prefer std::make_unique and std::make_shared to direct use of new.
  Item 22: When using the Pimpl Idiom, define special member functions in the
           implementation file.

CHAPTER 5  Rvalue References, Move Semantics, and Perfect Forwarding
  Item 23: Understand std::move and std::forward. 
  Item 24: Distinguish universal references from rvalue references. 
  Item 25: Use std::move on rvalue references, std::forward on universal
           references.
  Item 26: Avoid overloading on universal references. 
  Item 27: Familiarize yourself with alternatives to overloading on universal 
           references.
  Item 28: Understand reference collapsing. 
  Item 29: Assume that move operations are not present, not cheap, and not used. 
  Item 30: Familiarize yourself with perfect forwarding failure cases. 

CHAPTER 6  Lambda Expressions
  Item 31: Avoid default capture modes. 
  Item 32: Use init capture to move objects into closures. 
  Item 33: Use decltype on auto&& parameters to std::forward them. 
  Item 34: Prefer lambdas to std::bind. 

CHAPTER 7  The Concurrency API
  Item 35: Prefer task-based programming to thread-based. 
  Item 36: Specify std::launch::async if asynchronicity is essential. 
  Item 37: Make std::threads unjoinable on all paths. 
  Item 38: Be aware of varying thread handle destructor behavior. 
  Item 39: Consider void futures for one-shot event communication. 
  Item 40: Use std::atomic for concurrency, volatile for special memory. 

CHAPTER 8  Tweaks
  Item 41: Consider pass by value for copyable parameters that are cheap to 
           move and always copied.
  Item 42: Consider emplacement instead of insertion. 
Technical writing archeologists may wish to compare this TOC with the versions I showed on 18 March 2014, 5 April 2013, and 29 January 2013. I told you things would change!

Close-to-Final Item on noexcept

I posted drafts of my Item on noexcept on 31 March 2014 and 4 February 2014, so I felt obliged to show you the final-unless-I've-really-made-a-serious-mistake version. Here it is:
Let me know what you think. You've never been shy before, and I have no reason to think things will be different this time around :-)

Scott


Wednesday, August 13, 2014

Quick Test Needed for Boost.TypeIndex under Clang

I don't have Clang here (Windows ghetto, sorry), and I haven't been able to find an online Clang compiler that has the Boost headers available, so I'd be grateful if somebody would run the following program under Clang with the latest Boost and let me know (as a blog comment, so that people will know that the work has already been done) what the output is.

Thanks.

Scott


#include <iostream>
#include <vector>
#include <boost/type_index.hpp>

template<typename T>
void f(const T& param)
{
  using std::cout;
  using boost::typeindex::type_id_with_cvr;
  // show T
  cout << "T = "
       << type_id_with_cvr<T>().pretty_name()
       << '\n';
  // show param's type
  cout << "param = "
       << type_id_with_cvr<decltype(param)>().pretty_name()
       << '\n';

}

class Widget {};

std::vector<Widget> createVec()
{
  return std::vector<Widget>(5);
}

int main()
{
  const auto vw = createVec();
  if (!vw.empty()) {
    f(&vw[0]);
  }
} 

Thursday, July 24, 2014

Should move-only types ever be passed by value?

[For this post, I'm going to pretend that std::unique_ptr is a type, instead of a template, because the issue being examined is independent of what a std::unique_ptr points to.]

Suppose I want to pass a std::unique_ptr to a constructor, where the std::unique_ptr will be moved into a data member. The std::unique_ptr parameter thus acts as a sink. To the extent that we have enough experience with C++11 for wisdom about it to be conventional, said wisdom seems to be that the std::unique_ptr should be passed by value. In his GotW 91 solution, Herb Sutter argues for it. The High Integrity Coding Standard has it as a guideline. (It cites Herb's article as the source.) In his C++ Reference Guide, Danny Kalev argues for it. Many StackOverflow answers repeat this advice.

But recently Matthew Fioravante brought a StackOverflow question to my attention showing a problem resulting from declaring a std::unique_ptr by-value sink parameter, and later Matthew suggested that sink parameters of move-only types should be passed by rvalue reference. This is a very interesting idea.

Suppose you see this function signature:
void f(SomeType&& param);
What does this tell you about param? The fact that it's an rvalue reference tells you that it's a candidate to be moved from, and the usual expectation is that it will be. In other words, it's a sink parameter. Note that this is completely independent of param's type. Even without knowing anything about SomeType, we can conclude that param is a sink parameter.

If SomeType happens to be std::unique_ptr, nothing changes: param is still a sink. There's no need for a special rule for std::unique_ptrs that tells us to pass them by value to indicate that they're sinks, because we already have a way to unambiguously say that: pass them by rvalue reference.

Going back to the idea of a constructor moving a std::unique_ptr into a data member, this is what the code looks like using pass by value:
class Widget {
public:
  explicit Widget(std::unique_ptr ptr): p(std::move(ptr)) {}

private:
  std::unique_ptr p;
};
Now consider this calling code:
std::unique_ptr up;

Widget(std::move(up));
What's the cost of getting up into p? Well, the parameter ptr has to be constructed, and the data member p does, too. Each costs a move construction, so the total cost is two move constructions (modulo optimizations).

Now consider the same thing using pass by rvalue reference:

class Widget {
public:
  explicit Widget(std::unique_ptr&& ptr): p(std::move(ptr)) {}

private:
  std::unique_ptr p;
};

std::unique_ptr up;

Widget(std::move(up));
Here, only the data member p will be constructed, so the total cost is only one move construction.

Unless I'm overlooking something, passing sink parameters of type std::unique_ptr by value is inconsistent with our usual idiom for expressing the idea of a sink parameter (i.e., to pass by rvalue reference), and it's less efficient, too. My sense is that the conventional wisdom regarding sink parameters of type std::unique_ptr is all messed up.

Which leads to the question: how did it get messed up?  I believe what happened was that people noticed that for maximal efficiency when passing lvalues and rvalues of a particular type that needed to be copied inside the function, you needed to either overload on lvalue references and rvalue references, or you needed to pass by universal reference. Both approaches have problems (overloading doesn't scale to multiple parameters, and universal references suffer from the shortcomings of perfect forwarding, lousy error messages, and sometimes being too greedy). For cheap-to-move types, people found, you can use pass by value with only a modest efficiency loss, and the conventional wisdom, in large part based on a David Abrahams' blog post, "Want Speed? Pass by value", came to embrace that idea.

The thing is, for move-only types like std::unique_ptr, you don't need to worry about dealing with lvalues, because lvalues get copied, and move-only types aren't copyable. So there's no need to overload for lvalues and rvalues, hence no scalability problem for multiple parameters. Which means that the motivations for replacing pass by reference--which is what the conventional wisdom from C++98 always dictated--with pass by value don't exist for move-only types.

My feeling is that Matthew Fioravante may well have hit the nail on the head here: there is no reason to use by-value parameters to express "sinkness" for move-only types. Instead, the usual rule of passing sink parameters by rvalue reference should apply.

The special case of considering the use of pass by value for always-copied parameters really only applies to types that are both copyable and movable, and only in situations where overloading and the use of a universal reference is not desired.

What do you think? Is there ever a time where move-only types should be passed by value?

Scott

Saturday, July 19, 2014

Free Excerpt from Draft EMC++ Now Available

O'Reilly has made the TOC, Introduction, and first chapter ("Deducing Types") from the draft version of Effective Modern C++ available for free download. That's roughly the first 40 pages of the book, including Items 1-4. The PDF is available here.

I hope you enjoy this sample content, but I can't resist reminding you that this is from a draft manuscript. The final version will be better.

As always, I welcome suggestions for how this material can be improved.

Scott

Friday, July 18, 2014

Is the non-pointer syntax for declaring function pointer parameters not worth mentioning?

In my view, one of the most important favors a technical writer can do for his or her readers is shield them from information they don't need to know. One of my standard criticisms of authors is "S/he knows a lot. S/he wrote it all down." (Yes, I know: the "his or her" and "s/he" stuff is an abomination. Please suffer in silence on that. There's a different fish I want to fry in this post.)

I try not to commit the sin of conveying unimportant information, but one of the hazards of spending decades in this business is that you learn a lot. After a while, it can be hard to evaluate what's worth knowing and what's best left unsaid. With that in mind, the current draft of Effective Modern C++ contains this passage (more or less):
Suppose our function f can have its behavior customized by passing it a function that does some of its work. Assuming this function takes and returns ints, f could be declared like this:
     void f(int (*pf)(int));         // pf = "processing function"
It’s worth noting that f could also be declared using a simpler non-pointer syntax. Such a declaration would look like this, though it’d have the same meaning as the declaration above:
     void f(int pf(int));            // declares same f as above

One of my reviewers argues that the second way of declaring a function pointer parameter is not only not worth noting, it's not worth knowing. That means I should omit it. But I find myself reluctant to do that. I like the fact that function pointer parameters can be declared without throwing in additional parentheses and figuring out which side of the asterisk the parameter name has to go on. But maybe that's just me.

So what do you think? Should I jettison the aside about the asterisk-free way to declare the parameter pf, or should I keep it?

Thanks,

Scott

Friday, July 11, 2014

Draft Version of Effective Modern C++ Now Available

A full draft of Effective Modern C++ is now available through O'Reilly's Early Release program and Safari Books Online's Rough Cuts program.

The cover design for the book has changed. I'm the common crane no longer. Now I'm the fruit dove. I'm not sure what to make of this. Is going from common to fruity an upgrade?

My editor explained the reason for the avian switcheroo:
Since we are going full color for the book, the animal on the cover is going to be full color. The common crane isn't super colorful so we are switching to the fruit dove because it is filled with color.
During the early release it will be in black and white and the color will make its appearance soon.
The big information here is that O'Reilly has decided to print the book in full color. That's a first for me, and it means that the print book will display the different colors I originally put into the manuscript with only digital publishing in mind.

Well, the print book might display those colors. I'm not preparing the formatted content for this book. Instead, I'm working on a manuscript into which I put all the logical information that O'Reilly will need to produce the appropriate formatting for the files that will ultimately be used for publication and distribution. These will include PDF, ePub, and Kindle-compatible .mobi. So text in my manuscript that's red or blue or tan may not show up in those colors in the final product.

However, those colors will show up in the Early Release version of the book, because that document is simply a PDF print-out of the manuscript I'm working on. Please don't assume that the Early Release document reflects what the final book will look like. Even if O'Reilly chooses to use the colors I selected, they'll definitely clean up many aspects of the page layout that I designed to be seen only by me and my technical reviewers.

The Early Release book reflects the state of the manuscript as of about six weeks ago. Since then, I've read through the several hundred comments I got from my reviewers. Of the 315 pages in the manuscript, only 91 survived unscathed. That means that over 70% of the pages in the Early Release version have problems (that I know about). Some issues are small, e.g., misspellings or grammatical errors. Others involve important technical shortcomings that I'll have to think hard about how to address. To date, my favorite error report concerns a page in the manuscript where I managed to mis-translate a simple C++14 lambda expression into a corresponding C++11 lambda or call to std::bind in three different ways. I got it wrong. Three times. In three different ways. On one page. 

The draft book is available either as a standalone purchase or though Safari Books Online. At Safari, the book is currently associated with yet a third cover design (a hummingbird), but I believe we're going with the dove. Yet maybe not. I'm less in the loop on this stuff than you might think an author would be.

If you read the draft Items I posted earlier (here and here), you may be disappointed or even alarmed to see that the Items in the Early Release draft are essentially the same as what I posted. Relax. In the past few months, I've been focusing on finishing a full draft of the book, so work on revising existing Items got pushed to the back burner. I'm tending to those burners now, so the final versions of the Items will include changes in my thinking based on the comments I've received. 

I'm a little nervous about making this draft available. My normal modus operandi is to make things as good as I can before I publish them. What you're seeing in this case is a draft that was as good as I could make it before I sent it out for technical review. Based on the comments of my reviewers, I'd say it's not in bad shape, but there are a lot of places (some 70% of them) that need work. If you see opportunities for improvement, feel free to let me know about them, either by using the O'Reilly or Safari online errata reporting mechanisms or by sending me email (smeyers@aristeia.com). I don't expect to have time to respond to or to even acknowledge every report that comes in, but I assure you that I'll read them all and do my best to incorporate them into the final version of the book.

Scott

Wednesday, June 25, 2014

Another New Video

When I recently posted about five new videos, I didn't realize that Yandex had published my 11 June presentation at the Moscow C++ Party. There I gave a talk, "Why C++ Sails When the Vasa Sank," which is almost a bookend for the talk I gave at the D Conference last month.
"Why C++ Sails..." is my attempt to explain why C++, despite its size and complexity, continues to be a successful and important programming language.

My comments at the beginning of the video probably make more sense if you know that I had just been introduced in Russian, which is a language I don't know.

I think the double-wide format Yandex published that shows the speaker on one side of the video and the slides on the other works really well. That's overlooking, of course, that the speaker in this case is me. Yeesh. Talk about a need for a makeover...

I realize that other outlets (e.g., the isocpp.org blog) have already reported the existence of this video, and it's got at least two independent threads at reddit (here and here), but I wanted to hold off announcing it until it had been added to my web site's Online Videos and Past Talks pages, including links to downloadable video and presentation materials. Things have been a bit backed up here, hence the delay.

Scott