Here’s some information on it, it seems that even when we were on the gold standard audits were not allowed.
September 20, 1974: The inspection by Members of Congress on September 23, 1974, of U.S. gold stocks stored at the Fort Knox Bullion Depository marks a unique departure from the long standing and rigidly enforced policy of absolutely no visitors.
www.usmint.gov
According to this link, and it is a government link, a creditable audit was done by Congress with journalists in attendance in 1974. This is significant for several reasons.
Obviously it requires you to trust Congress and the media which none of us do, but importantly 1974 is three years after France forced Nixon to finally kill the last vestiges of the gold standard; allegedly temporarily.
There are a lot of details here that I haven’t looked into yet but apparently France demanded to exchange their dollars for gold as was allegedly their right under the Breton Woods agreement (there were two of them and I don’t know the difference).
Nixon flat out told them no. In a very French move they had sent a warship to make the demand for some reason. The speculation was that we didn’t have the gold (which I think was true in part. We didn’t have the gold to cover all the dollars that could come looking for it). This audit was done to quiet those fears.
We were supposed to go back on the limited gold standard at some unspecified point but obviously that didn’t happen.