Thursday, August 5, 2021

Timing Short Durations

 I don't have time for a long post (HA!), but I wanted to add a pointer to https://github.com/fordsfords/nstm ("nstm" = "Nano Second Timer"). It's a small repo that provides a nanosecond-precision time stamp portably between MacOS, Linux, and Windows.

Note that I said precision, not resolution. I don't know of an API on Windows that gives nanosecond resolution. The one Microsoft says you should use (QueryPerformanceCounter()) always returns "00" as the last two decimal digits. I.e. it is 100 nanosecond resolution. They warn against using "rdtsc" directly, although I wonder if most of their arguments are mostly no longer applicable. I would love to hear if anybody knows of a Windows method of getting nanosecond resolution timestamps that is reliable and efficient.

The best way to measure short duration things is to string together a whole lot of them and take an average. One advantage of this approach is that taking a timestamp itself takes time; making the measurement changes the thing you are measuring. So amortizing that cost over many iterations minimizes its influence.

But sometimes, you just need to directly measure short things. Like if you are histogramming them to get the distribution of variations (jitter).

I put some results here: https://github.com/fordsfords/fordsfords.github.io/wiki/Timing-software


Friday, July 9, 2021

More Perl "grep" performance

In an earlier post, I discovered that a simple Perl program can outperform grep by about double. Today I discovered that some patterns can cause the execution time to balloon tremendously.

I have a new big log file, this time with about 70 million lines. I'm running it on my newly-updated Mac, whose "time" command has slightly different output.

Let's start with this:

time grep 'asdf' cetasfit05.txt
... 39.388 total

time grep.pl 'asdf' cetasfit05.txt
... 21.388 total

About twice as fast.


Now let's change the pattern:

time grep 'XBT|XBM' cetasfit05.txt
... 24.787 total

time grep.pl 'XBT|XBM' cetasfit05.txt
... 18.940 total

Still faster, but nowhere near twice as fast. I don't know why 

Now let's add an anchor:

time grep '^XBT|^XBM' cetasfit05.txt
... 25.580 total

time grep.pl '^XBT|^XBM' cetasfit05.txt
... 3:08.25 total

WHOA! Perl, what happened????? 3 MINUTES???

My only explanation is that Perl tries to implement  a very general regular expression algorithm, and grep implements a subset, and that might cause Perl to be slow in some circumstances. For example, maybe the use of alternation with anchors introduces the need for "backtracking" under some circumstances, and maybe grep doesn't support backtracking. In this simple example, backtracking is probably not necessary, but to be general, Perl might do it "just in case". (Note: I'm not a regular expression expert, and don't really know when "backtracking" is needed; I'm speculating without bothering to learn about it.)

Anyway, let's make a small adjustment:

time grep.pl '^(XBT|XBM)' cetasfit05.txt
... 17.910 total

There, that got back to "normal".

I guess multiple anchors in a pattern is a bad idea.


P.S. - even though this post is about Perl, I tried one more test with grep:

time grep 'ASDF' cetasfit05.txt
... 26.132 total

Whaaa...? I tried multiple times, and lower-case 'asdf' always takes about 40 seconds, and upper-case 'ASDF' always takes about 27 seconds. I DON'T UNDERSTAND COMPUTERS!!! (sob)

Wednesday, March 17, 2021

Investment Advice from an Engineer

 I have some financial advice regarding investing in stocks. But be aware that, while I am pretty wealthy, the vast majority of my wealth came from salary from a good-paying career. You will *NOT* get rich off the stock market by following my advice.


THE ADVICE

Put money every month into an U.S. exchange-traded fund that is very broad market. Like SPDR. (I prefer VTI because it is even broader and has lower fees, but the differences aren't really that big). The goal is to keep buying and never selling, every working month of your life until you retire. (I don't have retirement advice yet.)

If the market starts going down, do *NOT* stop buying. In fact, if you can afford it, put more in. Every time the market goes down, put more in. The market will go back up, don't worry. 

The same cannot be said for an individual stock -- sometimes a company's stock will dive down and then stay down, basically forever. But the market as a whole won't do that. A dive might take a few days to recover, or might take a few years to recover. But it will recover. DON'T sell a broad fund if the market is going down. BUY it.


AND THIS WILL GET ME RICH?

No. It will give you the highest probability of a good return. Back when I was a kid, I was told to put money into a bank savings account. That was poor advice then, and is terrible advice now with interest rates close to zero. Putting money into a guaranteed bank account is guaranteed to underperform inflation. I.e. it will slowly but surely lose value.

Instead, tie your money to the overall market. The broad market has its ups and downs, but if you stick with it for a while, the overall trend is higher than inflation. 


WHAT IF I WANT TO GET RICH QUICK?

Well, you could buy a lottery ticket. That will get you rich the quickest. Assuming you win. Which you won't. Or you could go to Vegas and put a quarter into a slot machine.

But you're smarter than that. You know the chances of getting rich off of lotteries or gambling are too low. You want something that has some risk, but which will probably make you rich. And you're thinking the stock market.

Your best bet? Find a company currently trading for a few dollars a share, but is definitely for sure going to be the next Apple or Microsoft. One tiny problem: if it is definitely for sure going to be the next Apple or Microsoft, the stock price won't be a few dollars a share. It will be priced as if it already IS the next Apple or Microsoft. This is because there are tens of thousands of smart traders out there who are spending a HELL of a lot more time than you are trying to find the next big company. For a "sure thing" company that is already publicly traded, those tens of thousands have already bid up the price.

The only real chance for you to get in on the ground floor is to find a garage shop start-up, and invest in them. I have a rich friend who has made a lot of money doing exactly that. For every ten companies he invests in, nine go bankrupt within 5 years. And one goes off like a rocket.

That's how you do it. And unfortunately, you have to start out pretty rich to make this work. And you need to research the hell out of startups, and have a good crystal ball.

The only other way to do it is to find a company that the tens of thousands of smart traders thinks will NOT be a big deal, but you believe something they don't. Microsoft was publically traded at a low price for many years. The tens of thousands of smart traders in the 70s didn't think home computers would become a thing. And the few people who believed otherwise became rich.

The problem is that belief usually doesn't work very well at predicting success.

Look at BitCoin. I know several people who have made huge amounts of money on BitCoin. They did their initial investments based on a belief. A belief that BitCoin would become a real currency, just like dollars and euros; that people would be use BitCoin every day to buy and sell everything from gasoline to chewing gum. They looked at the theories of money and the principles of distributed control, and thought it would definitely for sure replace national currencies.

Those friends of mine were wrong. They made money for a completely different reason: speculators. Speculators buy things that are cheap and they think will go up in price. If enough speculators find the same thing to buy, the prices *does* go up. And more speculators jump in. BitCoin is a big speculative bubble, with almost no intrinsic value. (And yes, I know BitCoin is more complicated than that. But I stand by my 10,000-foot summary.)

Now don't get me wrong. Successful speculators DO become rich. Who am I to argue with success? But getting rich off of speculation is all about timing. You have to find the next big thing before most of the other speculators do, and then jump back out *before* it has run its course. Will BitCoin come crashing back down? Not necessarily. If enough people simply *believe* in it, it will retain its value. My own suspicion is that it will eventually crash but what do I know? I thought it would have crashed by now.

That's also what happened with GameStop. A group of Reddit-based speculators decided to pump up the price of a company. If you were in on it from the start, you probably made a ton of money. But once it hit the news, it was too late for you to get rich off of it. The best you could hope for was to make a little money and then get out FAST. But most people who jumped into GameStop after it had already made the news ended up losing money.

(BTW, "pump-and-dump" is against the law. I will be interested to find out if any of the Reddit-based traders get in trouble.)

Anyway, I know of many people who have taken a chance on a stock, and made some money. But they didn't get rich. And if they keep trying to replicate their early success, they'll end up winning some and losing some. And if they're smart, and work hard at it, they may out-perform the overall market in the long run. But remember - those tens of thousands of smart traders are also trying to out-perform the overall market. For you to do it repeatedly for many years probably requires expertise that those smart traders don't have. And you don't get rich quick this way, you just make some good money.


WHAT IF I JUST WANT TO PLAY THE MARKET

(shrugs) Everybody needs a hobby. I have a friend who goes to Vegas once a year. Sometimes he comes back negative, sometimes positive. He has absolutely no illusion that he will get rich in Vegas. He assumes he will either make a little or lose a little. And he has fun doing it. There's nothing wrong with spending money to have fun.

If you play the stock market as a game, where you aren't risking your financial future, then more power to you. But I knew one person who had to stop for his own emotional well-being. He started feeling bad every time he lost some money because he should have invested less, but also felt bad when he made money because he should have invested more. Overall he made money, but he had so much anxiety doing it that he decided it wasn't worth it.

Sunday, March 14, 2021

Circuit simulation

 I've been playing with designing simple digital circuits this weekend. Since my breadboards are not with me at the moment, I decided to look for circuit simulators.

Here's a nice comparison of several: https://www.electronics-lab.com/top-ten-online-circuit-simulators/

Before I found that comparison site, I tried out https://www.circuitlab.com/ and even threw them money for a month's worth. And I can say that I've gotten that much worth of enjoyment out of my tinkering this weekend, so money well-spent. But I knew that I didn't want to keep shelling out every month (I don't do digital design that much), and there was no way to export the circuits in a way that I could save them. So I kept looking.

Here's my CircuitLab home: https://www.circuitlab.com/user/fordsfords/

I haven't tried all the choices in the "top ten" list, but I did try the "CircuitJS1" simulator maintained by Paul Falstad and Iain Sharp. See https://github.com/sharpie7/circuitjs1 It isn't quite as nice as CircuitLab, but it's hard to argue with free, especially given my infrequency of use.

CircuitJS1 doesn't host users' designs. In fact, they don't integrate well with any form of storage. You can save your design to a local file, but the simulator doesn't do a good job of remembering file names. It presents you with a link containing a file name of the form "circuit-YYYYMMDD-HHMM.circuitjs.txt". You can save the linked contents with your own file name, but the next time you go to save, it obviously won't remember that name since it was a browser operation. All of this will make it a little inconvenient and perhaps error-prone to manage different projects. If I were doing a lot of hardware work, I would probably choose something else. But for occasional fiddling, this is fine.

Here's a simple state machine that checks even/odd parity of an input bit stream: http://www.falstad.com/circuit/circuitjs.html?startCircuitLink=https://www.geeky-boy.com/test.circuitjs.txt

If I want to make anything public, I'll make them as github projects.

Friday, January 8, 2021

Racism in America

 As my readers have no doubt noticed (all 2 of you!), I keep this blog pretty technical, without a lot of politics. And I intend to keep it that way ... for the most part. But occasionally I will let my politics peek out.

Yeah, you're expecting me to talk about the events in Washington DC in January, 2021. I might post about that some day, but I'm nowhere near ready yet.

No, I'm going to talk about a class I took last fall. See https://www.rootstorevolution.org/courses

These are left-leaning classes that not only teach history, they also encourage and facilitate activism. Their focus is on racism, but touch on other "isms" as well. I learned a heck of a lot of history that wasn't covered very well back when I went to high school. The material is well-researched and well-sourced. I consider myself a better person for having participated.

The bottom line is that it isn't enough to be "not racist". We have to be "anti-racist".

The classes are not cheap. As of this writing, they are $200 a pop. (And worth it, in my humble opinion.) That said, the class organizers don't want cost to be a barrier to participation, and are willing to make adjustments. Plus, I am willing to kick in $100 for anybody who comes to them from my recommendation. Tell 'em fordsfords sent ya. (-:

Anybody who wants more "informal" information on the classes, send me an email.

Steve