Reverse DNS is the hostname that an IP address answers with. Forward DNS says mail.yourdomain.com.au points at an IP. Reverse DNS says that IP points back at mail.yourdomain.com.au. Mail servers check this pair. If they do not match, messages from the VPS are more likely to be rejected or filed as spam.
You set it on the VPS service in the client portal. It is not the same screen as the nameservers for shared cPanel hosting.
Create the forward record first
Pick a hostname you control, such as vps.yourdomain.com.au or mail.yourdomain.com.au. At the DNS host for that domain, add an A record for that name and point it at the VPS IP address. Find the IP on the instance details in the client portal, or in the email sent when the server was provisioned.
Wait until that name resolves to the VPS IP. Reverse DNS that names a host which does not point back at the same address will still fail mail checks.
Set reverse DNS in the portal
- Log in to the client portal and open the Linux VPS.
- Under Service Management, open Reverse DNS.
- Find the IPv4 address in the list. The table shows the protocol, the IP, and the current reverse DNS value.
- Click Update.
- In Update DNS Record, enter the hostname. Use the name only, without
https://and without a path. - Save. The portal confirms when the reverse DNS record has been updated.
IPv6 is a separate row. Use Create IPv6 Reverse DNS if you send mail over IPv6 and need a matching hostname there. You can remove an IPv6 record with Delete. Deleting the IPv4 record is not the usual fix. Update it to the right hostname instead.
Check that the pair matches
From your own computer, look up the hostname and the IP.
dig +short vps.yourdomain.com.au
dig +short -x your.vps.ip.address
The first command should print the VPS IP. The second should print the hostname you saved, with a trailing dot. If the second is empty, give the change some time and check that you updated the IP that mail actually leaves from.
Reverse DNS does not point a website at the server. The website still needs an A record, which is covered in How to Point a Domain at Your Linux VPS. Shared hosting nameservers stay on the cPanel nameserver guide. Those nameservers send the domain to shared hosting, not to this VPS.