The naysayers will say Buffett was too early then, and now