According to the Yoga Sutras, a collection of aphorisms on the philosophy called yoga, yoga is defined as:
Correct! The physical practice of yoga, also known as asana, is a tool to find that stillness.
There's actually 8 limbs of yoga, including asana the physical practice. Each limb helps you find the stillness of the chatter of the mind
So, the downward dogs and chaturangas are part of yoga, but not the only aspect of it!
All of the things you might find in a yoga class like the physical postures or breath work are all about helping you find that stillness of the mind.
Finding stillness can be HARD. The physical practice is there to essentially work your body out so it is easier to find stillness like in savasana.