Advanced Networking Guide
Classic vs Advanced Networking in the Melbourne Research Cloud
Instructions for creating your own private network
- Create security group and rules
- Add security group to instance
- Remove security group from instance
- Re-add security group to instance
Isolating from public internet
Introduction
All projects could get access to three networks:
qh2-uom, which is directly exposed to the public internet, requires Cybersecurity approval;qh2-uom-internal, which can only be reached from inside the University of Melbourne (including via VPN);Classic Provider, which is currently equivalent toqh2-uom-internal.
Associating an instance with qh2-uom is required if you plan to offer a service such as a website which needs to be accessed from everywhere.
It can also be convenient for logging into your instance while away from campus, without needing to use the VPN.
However, instances exposed to the public internet in this way will be subject to immediate and unrelenting attack by botnets and hackers. Our default configuration for your virtual machines (e.g. no password logins) follows industry best practices and can be considered quite secure; however, once you start customising the configuration or exposing more services, this can weaken the security considerably.
Each year we have several security incidents resulting from our users exposing insecure services to the public internet. This is always embarrassing and inconvenient for everyone involved. You should only expose a service to the public internet if you have the systems administration skills required to keep it secure and up-to-date.
Using the qh2-uom-internal network only exposes the instance to a private network inside the University of Melbourne.
Access to the instance is now only available from on-campus or via the staff or student VPN.
This completely eliminates the threats posed by the public internet.
In addition to qh2-uom and qh2-uom-internal, it is also possible to create your own private network, complete with virtual routers and load balancers.
For example, you might want to create a cluster where some services are hidden away inside a private network while other services are exposed to the public internet.
Such non-default networking is not automatically granted to new projects. Instead, you need to request Advanced Networking quota. This quota can be requested either for a new project, or for an existing project via Amend/Extend request.
Various types of additional quota are available under Advanced Networking, including load balancers, private networks, virtual routers, and "floating" IPs which can be moved from one instance to another:
Advanced Networking key terms
Fixed vs Floating IP Addresses
By default, when you launch an instance, it is automatically assigned an IP address. When the instance is deleted, you lose that IP address. At this point, the IP address gets recycled back into a pool of IP addresses available for any MRC project to claim. These default IP addresses are called Fixed.
If you know you will need an IP address that you can keep using even after an instance is deleted, you can use Floating IP addresses. They are called this because they can "float" from one instance to another. They are not available by default; you need to request Floating IP address quota through the allocation form.
The primary use of floating IP addresses is if you need to access a third-party site which only permits access based on a whitelist.
If your goal is simply to provide a consistent way to reach your instances, it is better to use DNS instead of floating IP addresses. DNS records for your project are now easy to manage through the MRC dashboard.
Private Networks
Many networked devices we're familiar with in day-to-day life are not directly connected to the internet. For example, whenever you're using wifi, you first connect to the wifi router, and then the router mediates your connection to the rest of the internet.
When you are relying on a router to mediate your internet access this way, you are using a private network. Devices behind the same router can often see each other, but the external internet doesn't see anything except the router.
This has several benefits. It is convenient because you don't have to worry about setting up a separate internet connection for every device, everything can just be delegated to the router. It is secure because by default your devices can't be attacked by the myriad threats on the public internet; the external internet only sees what the router exposes.
Subnets
Networks are partitioned into one or more sets of IP addresses referred to as subnets. When defining a private network, you must also define a subnet to specify the range of IP addresses you want available in the network.
You might have multiple subnets if you have a complicated setup where some of your machines should not have direct access to other machines on the same network. Most of the time when you create a private network on the MRC, it will just have one subnet.
Subnets are specified using CIDR notation, which combines an IP address with a number of significant bits. It is outside the scope of this documentation to fully describe CIDR notation; instead, we will make a small number of suggestions, and refer you to Wikipedia for more details.
Typical CIDR ranges for private networks are:
10.0.0.0/8172.16.0.0/12192.168.0.0/16
The smaller the number after the slash, the more IP addresses will be available in the subnet.
Each increment halves the size of the subnet.
Therefore, 10.0.0.0/8 covers over 16 million IP addresses, while 192.168.0.0/16 has fewer than 70,000.
A larger subnet isn't necessarily better.
If you only have quota for a small number of instances, a large subnet will simply result in more scattered IP addresses.
For most research cloud projects, you will probably realise that a /24 or even /27 network will be more to your tastes.
Classic vs Advanced Networking in the Melbourne Research Cloud
Classic Networking:
By default, compute instances are assigned a fixed IP address and are connected directly to the internet via a public network located at the instance’s availability zone.
Advanced Networking
If you have Advanced Networking quota, you can create your compute instances in a private network with a virtual router. The router is connected to the internet via a public IP address. Internet traffic can flow from the private network through the router to the public internet.
Requesting Quota
If your project requires advanced networking, please go to the Allocations -> My Requests tab in the dashboard and open an Amend/Extend allocation request. Look for the "Advanced Networking" section and fill it in.
Note that floating IPs also require a private network and a router. You can have a private network and router without floating IPs, though.
You can confirm your quota in the Project -> Overview section of the Dashboard; it might look a bit like this:
You can refer to the Network -> Network Topology tab for a visual representation of your research cloud networks.
Note that it is not a good idea to mix Classic Networking and Advanced Networking when creating new instances. The instance will end up with two default routes, which breaks communication with the instance.
Instructions for creating your own private network
Set up the network itself
Once you have been approved with Advanced Networking quota, you can work through this example about how to start using it in your project.
First, select the desired project from the Nectar dashboard and navigate to the Network -> Networks tab:
Click the Create Network button. From the Create Network dialog, enter a name for your private network. The other options should usually be left at their default values.
Click Next to go to the Subnet tab:
Give the subnet a name and an IP address range in CIDR notation, as described above.
You can leave the gateway IP address field blank and it will be automatically set as the first IP address in the range.
For example, if you specify 192.168.0.0/27 in the previous field, the gateway will default to 192.168.0.1.
If you disable the gateway, the next tab Subnet Details will not be available. In this case, the instance operating system will also not have a default route set.
The entries in the Subnet Details tab can usually be left at the default values. Click Create to instantiate your new network.
Create a Router
Usually when you create a private network, you should also create a router for it. There are several reasons for this:
- Routers act as a gateway for instances in the private network to retrieve information from the external internet.
- Similarly, a router is required for instances inside the private network to reach the OpenStack metadata service (
169.254.169.254). The metadata service is used when creating an instance for tasks such as setting the hostname and injecting your ssh public key. Failure to communicate with the metadata service may result in an impaired or non-accessible instance. - By keeping multiple instances behind a router, fewer public IP addresses are required. We only have a finite number of expensive public IP addresses, so it is appreciated when you can conserve this resource.
- Routers shield your instances from botnet attacks originating from the public internet. The modern internet is rife with powerful botnets which relentlessly hammer every machine attached to the public internet. Effective firewalls and hardened configuration can ameliorate the risk, but hiding sensitive services (such as databases) inside a private network offers an additional layer of security.
- If you have multiple subnets in a single private network, a router can also be used to direct traffic between them.
Note that these virtual router and attaching it to an external network does not necessarily provide a way into the instance from the internet, only a way out. If you want to create a way into the instance, we recommend using floating IP's (subject to quota allowance), as instructed below.
Navigate to the Network -> Routers tab:
Click the Create Router button.
In the Create Router dialog, enter the name of your router.
Select melbourne as the external network.
This will attach a publicly routed IP (not to be confused with Floating IP) to your router and allow outbound communication from any instance that joins your gateway-enabled private subnets.
When you are ready, click the Create Router button.
Associate the router with a network
Once the router has been created you must attach it to the private network of choice. Make sure you are on the Routers tab, then click on the router you just created:
Then select the Interfaces tab and click Add Interface.
From the Add Interface dialog, select the Subnet you created earlier from the drop down box. If you specified a Gateway IP address earlier when creating the subnet, the router should also be given that same IP address here. Once completed, click Submit.
Your private network is now ready for use by compute instances.
Using the private network
Adding a new instance to the network
In order for a compute instance to communicate on a private network it must be attached to an interface on that network. This section describes attaching a private network to an instance at instance launch time.
From within the Launch Instance window in the dashboard select the Networking tab.
Add your private network to Allocated Networks using the ↑ button.
We will remove the Classic Provider network in this example as we intend to use floating IPs for inbound communications.
Attaching a floating IP to an instance
In order for a floating IP to be attached to your instance, your instance needs to be part of a private network which contains a gateway-enabled subnet. This includes the presence of a virtual router so that traffic can egress to the external internet.
From the Compute -> Instances tab, find the instance from the previous step, and use the actions menu to click Associate Floating IP.
Select a floating IP address from the list and ensure your instance port is selected in the Port to be associated field.
If you see "No floating IP addresses allocated", click the + (plus) icon;
this will take you to a new window where you can claim a floating IP address for use in your project.
When finished, click Associate.
Verify the floating IP is now present in the Dashboard; you may need to refresh the page.
Once you are satisfied, you may try logging into your instance via the floating IP address.
Making Changes
It is not usually a good idea, but if you need to, you can attach and detach interfaces on existing virtual machines. These actions can both be performed using the instance's action menu.
Likewise, a floating IP address can be disassociated from an instance from the actions menu.
Security groups
Security groups act as a virtual firewall for instances, controlling both inbound and outbound network traffic. Each security group is a collection of rules that define the type of traffic and direction (either incoming or outgoing) that is allowed or denied to pass through to an instance. These rules can specify various parameters, such as protocol (TCP, UDP, ICMP), port range, and the source or destination IP range that can be used to filter traffic. When interpreting security group rules, it's crucial to understand that they are evaluated collectively as a set, with the default policy to deny all traffic that does not explicitly match any rule. Each rule in the group acts as a whitelist condition, where only the specified traffic is permitted and all others are implicitly rejected. This way, a user can create granular policies tailored to the specific needs of each instance.
Create security group and rules
Creating a security group is simple. Under the Network tab on the left, click on Security Groups and then on Create Security Group. For demonstration purposes, we'll create a security group to allow incoming SSH traffic from the public internet.
By default, 2 rules would be associated with a newly created security group - one which allows all egress (outgoing) traffic for IPv4 addresses from the instance to the public internet and the other which does the same but for IPv6 addresses. Notice how there's no rule for incoming traffic. This is done on purpose to implicitly deny all incoming traffic as part of good security hygiene.
Clicking on Add Rule would allow you to add rules to your security group. The reason behind the values in all the fields in the screenshot below is as follows -
- Rule - Custom TCP Rule - because SSH is based on TCP
- Description - optional but good practice to describe what your rule exactly does/is meant for
- Direction - Ingress - because we want incoming SSH traffic from the public Internet
- Open Port - Port - because we want to allow traffic only on a single port
- Port - 22 - because SSH uses port 22
- Remote - CIDR - because we'd like to specify an address range
- CIDR - 0.0.0.0/0 - because that specifies ALL traffic
- Ether Type - IPv4 (set by default) - because the address type being specified is of IPv4 type
The rules associated with a security group can always be added/deleted as required. There's no way to directly edit a rule on a security group, you will need to delete the rule. Remember that by default, all traffic is denied unless a rule whitelists/allows traffic. Clicking on the Manage Rules button on the Security Groups link under the Network tab on the left would give you the ability to add/delete rules associated with a security group.
Note: Allowing/denying traffic on an instance can work in 2 ways - either the security group associated with an instance can be added/deleted or the rules associated with a security group can be added/deleted, although in the latter approach if that particular security group is in use by other instances, it will affect traffic behavior on all those instances.
Add security group to instance
Now the security group can be added to an instance during it's creation or later on (see the re-add security groups section below). When launching a new instance, we can add this newly created security group when you reach the Security Groups section of the Launch Instance process.
To view the security groups (and the rules associated with them) attached to an instance, click on the Instance and the Overview tab would have this information displayed.
You should be able to SSH to your instance it whether you are on the University's VPN or on the University's network or on the public Internet (and of course you would also need to use the key associated with your instance for SSH).
ssh -i docnet-instance-key.pem ubuntu@45.113.234.167
...
...
...
ubuntu@docnet-instance:~$
Remove security group from instance
To remove the security group associated with an instance, go to the Instances page and click on the dropdown button beside Create Snapshot in the Actions column. Then click on Edit Security Groups.
On the window that pops up, you'll be able to add (+ button) and remove (- button) security groups. In this case, remove the security group we created by clicking on the - button and then click on Save. The instance's Overview page would then be displayed and the Security Groups section would indicate that the security group has been removed.
Now, you should NOT be able to SSH to your instance from the public Internet or from the University's network and the University's VPN.
~$ ssh -i docnet-instance-key.pem ubuntu@45.113.234.167
ssh: connect to host 45.113.234.167 port 22: Connection timed out
Re-add security group to instance
To re-add the security group to the instance, go to the Instances page and click on the dropdown button beside Create Snapshot in the Actions column. Then click on Edit Security Groups. This time, click on the + button associated with the security group and then click on Save.
Isolating from public internet
Transition networks - from qh2-uom to qh2-uom-internal
As mentioned above, instances should ideally be launched in the qh2-uom-internal network unless the instance needs to be accessed from the public Internet - in which case, the qh2-uom network should be used.
However, if your instance no longer needs to be accessible from the public Internet, then it needs to be transitioned to the private and internal qh2-uom-internal network.
To demonstrate how this transition can be done, we have an instance that has been launched in the qh2-uom network and can be accessed from the public internet (with the right security group). In the Actions column, click on the drop down arrow for the instance and click on Detach Interface.
In the Port field, there will be only one value - the IP address for that instance. Select it and click on Detach Interface to remove the instance from it's current network i.e. qh2_uom.
After detaching this (network) interface, the instance will NOT have an IP address associated with it (and security groups will be removed as well). This indicates that it has been removed from the qh2_uom network. Now, you need to add it to the qh2_uom_internal network. To do so, in the Actions column, click on the drop down arrow for the instance and click on Attach Interface.
Keep the default values except for the Network field - you must specify the network to which this instance needs to be a part of - which is qh2-uom-internal in our case. Click on Attach Interface.
The instance has now moved to the qh2_uom_internal network and will have a different IP address associated with it. Since a new interface was attached to the instance, only the default security group will be attached to it - so add the relevant security group(s) to the instance as per your use case. Keep in mind that the instance would now only be accessible from within the University's VPN since it's part of the qh2_uom_internal network (even if the rules in your security group allow traffic from the public Internet).
Remove floating IP
A floating IP associated with an instance allows it to be accessed from the public Internet and if this access is no longer needed, then the instance's floating IP needs to be disassociated with it.
In the Actions column, click on the drop down arrow for the instance and click on Disassociate Floating IP.
Select the floating IP to disassociate and then click on Disassociate.
This will remove the floating IP from the instance and now there's no way to access it (or rather the private network the instance is in) from the public Internet.
Using the public network
Transition networks - from qh2-uom-internal to qh2-uom public
If your instace uses a private IP and now needs to be accessible from the public Internet, then it needs to be transitioned to the public qh2-uom network.
To demonstrate how this transition can be done, we have an instance that has been launched in the qh2-uom-internal network and can only be accessed from the private network.
In the Actions column, click on the drop down arrow for the instance and click on Detach Interface.
In the Port field, there will be only one value - the IP address for that instance. Select it and click on Detach Interface to remove the instance from it's current network i.e. qh2_uom_internal.
After detaching this (network) interface, the instance will NOT have an IP address associated with it (and security groups will be removed as well). This indicates that it has been removed from the qh2_uom_internal network. Now, you need to add it to the qh2_uom network. To do so, in the Actions column, click on the drop down arrow for the instance and click on Attach Interface.
Keep the default values except for the Network field - you must specify the network to which this instance needs to be a part of - which is qh2-uom in our case. Click on Attach Interface.
The instance has now moved to the public qh2_uom network and will have a different IP address associated with it. Since a new interface was attached to the instance, only the default security group will be attached to it - so add the relevant security group(s) to the instance as per your use case. Note that your instance is now accessible world-wide, apply the necessary security groups to keep it secure and minimize cyber attacks.