You really shouldn't create Azure virtual machines with public IP addresses unless you need it to communicate with outside world. Instead use VPN or ExpressRoute to connect to the virtual machine using private IP address. To create Azure VM without a public IP address using Azure CLI use the following command:
az vm create --resource-group TEST --name TESTVM --public-ip-address ""
Yes, just use empty quotes to create your VM without a public IP address. If you do not specify --public-ip-address option then Azure CLI will create a new public IP address and assign it to your VM.
Another gotcha is that the above-mentioned command only works if you run in in the command line or bash. If you're running your Azure CLI commands from within PowerShell, then you need to wrap empty quotes one more time. So the command will look as such:
az vm create --resource-group TEST --name TESTVM --public-ip-address '""'
Apparently, PowerShell swallows empty quotes and never passes them to Azure CLI command, so if you don't wrap the quotes again in PowerShell window, you will end up a virtual machine with a public IP address, which is frustrating.
Hope this helps.
P.S. : If you need your virtual machine to have a public IP address, please do not forget to secure that VM using network security group and such…