I can watch the vac gauge drop when I push the pedal

Brake booster.

There's some kind of rubber disc inside that can fall loose. Never dealt with that but read plenty. Hopefully the others will fill in the blanks.

Edited out.

Edited September 1, 20197 yr by siteunseen

If the brake light comes, you have a pressure differential between the front and rear systems. See screenshot brake switch.

Could be a couple of things causing it, but if you can't find any brake fluid leaks at the wheels then it's likely the master cylinder leaking back into one of the reservoirs or back into the booster. That or air in one system.

Master cylinder? I think Mr. Maras is correct. 

Edited September 16, 20197 yr by conedodger