This is pretty much intentional, since it makes using the site much easier for "real" beta users. Your "public computer" point isn't wrong, but it's minor enough that we don't worry about it.
In particular let me stress this (you didn't touch on this issue, but it should be said anyway): This is not a security or privacy issue. Both the link to grant you beta access (the one you received in the email) and the "beta access granted" cookie are identical for everyone. They don't allow impersonating you or even just figuring out your identity. As long as you log out, you're good.
Now, on the off chance that this cookie remains somewhere so that someone can access the private beta despite not having been invited – well, maybe that's one more user who gets interested in the site, who knows? After all, you can also invite other people yourself using the "Invite Fellow Experts" widget on the homepage.
The main point of the private beta is to give committers and early adopters a chance to set the tone for the site, and to prevent an "empty restaurant syndrome" when the site is public, and the whole world comes and sees there's nothing here, and loses interest.
The private beta content is not considered secret. The Stack Exchange API actually makes it possible for people to read the content of a private beta from day one (but doesn't allow participation).
So in the rare case that someone gets access to beta a week or two too early – well, worst case is that they're bored and leave again. But the positive side of the system is that it makes things much easier for the committers, so we're willing to live with it.