跳转至

运维-02-网络服务-neutron

常用命令

网络创建

usage: openstack network create [-h] [-f {json,shell,table,value,yaml}]
                                [-c COLUMN] [--max-width <integer>]
                                [--fit-width] [--print-empty] [--noindent]
                                [--prefix PREFIX] [--share | --no-share]
                                [--enable | --disable] [--project <project>]
                                [--description <description>] [--mtu <mtu>]
                                [--project-domain <project-domain>]
                                [--availability-zone-hint <availability-zone>]
                                [--enable-port-security | --disable-port-security]
                                [--external | --internal]
                                [--default | --no-default]
                                [--qos-policy <qos-policy>]
                                [--transparent-vlan | --no-transparent-vlan]
                                [--provider-network-type <provider-network-type>]
                                [--provider-physical-network <provider-physical-network>]
                                [--provider-segment <provider-segment>]
                                [--tag <tag> | --no-tag]
                                <name>

创建一个不指定任何参数的网络

[root@10e129e169e140 ~]# openstack network create ops_test
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | UP                                   |
| availability_zone_hints   |                                      |
| availability_zones        |                                      |
| created_at                | 2018-08-20T09:31:11Z                 |
| description               |                                      |
| dns_domain                |                                      |
| id                        | e9dd85d7-6613-41f0-94c5-962e896be32b |
| ipv4_address_scope        | None                                 |
| ipv6_address_scope        | None                                 |
| is_default                | False                                |
| is_vlan_transparent       | None                                 |
| mtu                       | 1500                                 |
| name                      | ops_test                             |
| port_security_enabled     | True                                 |
| project_id                | 5e2214ea913a49babd5a4aab26e56ff8     |
| provider:network_type     | vlan                                 |
| provider:physical_network | tenant                               |
| provider:segmentation_id  | 2859                                 |
| qos_policy_id             | None                                 |
| revision_number           | 2                                    |
| router:external           | Internal                             |
| segments                  | None                                 |
| shared                    | False                                |
| status                    | ACTIVE                               |
| subnets                   |                                      |
| tags                      |                                      |
| updated_at                | 2018-08-20T09:31:13Z                 |
+---------------------------+--------------------------------------+

创建公网

[root@10e129e169e140 ~]# openstack network create ext-net2 --external --provider-network-type vlan --provider-segment 210 --provider-physical-network external --tag ops_test
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | UP                                   |
| availability_zone_hints   |                                      |
| availability_zones        |                                      |
| created_at                | 2018-08-21T02:16:33Z                 |
| description               |                                      |
| dns_domain                |                                      |
| id                        | 1e4bc15a-7d60-455b-bd18-d62ab1fe9a8d |
| ipv4_address_scope        | None                                 |
| ipv6_address_scope        | None                                 |
| is_default                | False                                |
| is_vlan_transparent       | None                                 |
| mtu                       | 1500                                 |
| name                      | ext-net2                             |
| port_security_enabled     | True                                 |
| project_id                | 5e2214ea913a49babd5a4aab26e56ff8     |
| provider:network_type     | vlan                                 |
| provider:physical_network | external                             |
| provider:segmentation_id  | 210                                  |
| qos_policy_id             | None                                 |
| revision_number           | 5                                    |
| router:external           | External                             |
| segments                  | None                                 |
| shared                    | False                                |
| status                    | ACTIVE                               |
| subnets                   |                                      |
| tags                      | ops_test                             |
| updated_at                | 2018-08-21T02:16:36Z                 |
+---------------------------+--------------------------------------+

验证

[root@10e129e169e140 ~]# openstack network list --external
+--------------------------------------+-------------+----------------------------------------------------------------------------+
| ID                                   | Name        | Subnets                                                                    |
+--------------------------------------+-------------+----------------------------------------------------------------------------+
| 1e4bc15a-7d60-455b-bd18-d62ab1fe9a8d | ext-net2    |                                                                            |
+--------------------------------------+-------------+--------------------------------------------------------

网络删除

[root@10e129e169e140 ~]# openstack network delete e9dd85d7-6613-41f0-94c5-962e896be32b

验证

[root@10e129e169e140 ~]# openstack network list |grep e9dd85d7-6613-41f0-94c5-962e896be32b

网络 agent 查询

usage: openstack network agent list [-h] [-f {csv,json,table,value,yaml}]
                                    [-c COLUMN] [--max-width <integer>]
                                    [--fit-width] [--print-empty] [--noindent]
                                    [--quote {all,minimal,none,nonnumeric}]
                                    [--sort-column SORT_COLUMN]
                                    [--agent-type <agent-type>]
                                    [--host <host>]
                                    [--network <network> | --router <router>]
                                    [--long]

以 host 查询 agent

[root@10e150e68e69 ~]# openstack network agent list --host 10e150e68e15
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
| ID                                   | Agent Type         | Host         | Availability Zone | Alive | State | Binary                    |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
| 02a381e8-907b-4f3a-bb1f-8d789a06f3da | DHCP agent         | 10e150e68e15 | nova              | :-)   | UP    | neutron-dhcp-agent        |
| 4f83d065-4de3-4487-b9c7-394b2aa68654 | L3 agent           | 10e150e68e15 | nova              | :-)   | UP    | neutron-l3-agent          |
| 86856f26-1405-419f-a440-b47aa65705fb | Metadata agent     | 10e150e68e15 | None              | :-)   | UP    | neutron-metadata-agent    |
| abacbeb7-b405-4bad-ac58-59b8e8c30f83 | Open vSwitch agent | 10e150e68e15 | None              | :-)   | UP    | neutron-openvswitch-agent |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+

查询 agent 详细信息

[root@10e150e68e69 ~]# openstack network agent show 02a381e8-907b-4f3a-bb1f-8d789a06f3da
+-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field             | Value                                                                                                                                                           |
+-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up    | UP                                                                                                                                                              |
| agent_type        | DHCP agent                                                                                                                                                      |
| alive             | :-)                                                                                                                                                             |
| availability_zone | nova                                                                                                                                                            |
| binary            | neutron-dhcp-agent                                                                                                                                              |
| configuration     | {u'subnets': 11, u'dhcp_lease_duration': -1, u'dhcp_driver': u'neutron.agent.linux.dhcp.Dnsmasq', u'ports': 76, u'log_agent_heartbeats': False, u'networks': 9} |
| created_at        | 2018-06-11 06:25:55                                                                                                                                             |
| description       | None                                                                                                                                                            |
| ha_state          | None                                                                                                                                                            |
| host              | 10e150e68e15                                                                                                                                                    |
| id                | 02a381e8-907b-4f3a-bb1f-8d789a06f3da                                                                                                                            |
| last_heartbeat_at | 2018-08-24 02:12:44                                                                                                                                             |
| name              | None                                                                                                                                                            |
| started_at        | 2018-08-14 08:53:32                                                                                                                                             |
| topic             | dhcp_agent                                                                                                                                                      |
+-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+

网络 agent 状态修改

关闭 agent

[root@10e150e68e69 ~]# openstack network agent set --disable 02a381e8-907b-4f3a-bb1f-8d789a06f3da

验证

[root@10e150e68e69 ~]# openstack network agent list --host 10e150e68e15
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
| ID                                   | Agent Type         | Host         | Availability Zone | Alive | State | Binary                    |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
| 02a381e8-907b-4f3a-bb1f-8d789a06f3da | DHCP agent         | 10e150e68e15 | nova              | :-)   | DOWN  | neutron-dhcp-agent        |
| 4f83d065-4de3-4487-b9c7-394b2aa68654 | L3 agent           | 10e150e68e15 | nova              | :-)   | UP    | neutron-l3-agent          |
| 86856f26-1405-419f-a440-b47aa65705fb | Metadata agent     | 10e150e68e15 | None              | :-)   | UP    | neutron-metadata-agent    |
| abacbeb7-b405-4bad-ac58-59b8e8c30f83 | Open vSwitch agent | 10e150e68e15 | None              | :-)   | UP    | neutron-openvswitch-agent |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+----

开启 agent

[root@10e150e68e69 ~]# openstack network agent set --enable 02a381e8-907b-4f3a-bb1f-8d789a06f3da

验证

[root@10e150e68e69 ~]# openstack network agent list --host 10e150e68e15
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
| ID                                   | Agent Type         | Host         | Availability Zone | Alive | State | Binary                    |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+---------------------------+
| 02a381e8-907b-4f3a-bb1f-8d789a06f3da | DHCP agent         | 10e150e68e15 | nova              | :-)   | UP    | neutron-dhcp-agent        |
| 4f83d065-4de3-4487-b9c7-394b2aa68654 | L3 agent           | 10e150e68e15 | nova              | :-)   | UP    | neutron-l3-agent          |
| 86856f26-1405-419f-a440-b47aa65705fb | Metadata agent     | 10e150e68e15 | None              | :-)   | UP    | neutron-metadata-agent    |
| abacbeb7-b405-4bad-ac58-59b8e8c30f83 | Open vSwitch agent | 10e150e68e15 | None              | :-)   | UP    | neutron-openvswitch-agent |
+--------------------------------------+--------------------+--------------+-------------------+-------+-------+-----

子网创建

usage: openstack subnet create [-h] [-f {json,shell,table,value,yaml}]
                               [-c COLUMN] [--max-width <integer>]
                               [--fit-width] [--print-empty] [--noindent]
                               [--prefix PREFIX] [--project <project>]
                               [--project-domain <project-domain>]
                               [--subnet-pool <subnet-pool> | --use-prefix-delegation USE_PREFIX_DELEGATION | --use-default-subnet-pool]
                               [--prefix-length <prefix-length>]
                               [--subnet-range <subnet-range>]
                               [--dhcp | --no-dhcp] [--gateway <gateway>]
                               [--ip-version {4,6}]
                               [--ipv6-ra-mode {dhcpv6-stateful,dhcpv6-stateless,slaac}]
                               [--ipv6-address-mode {dhcpv6-stateful,dhcpv6-stateless,slaac}]
                               [--network-segment <network-segment>] --network
                               <network> [--description <description>]
                               [--allocation-pool start=<ip-address>,end=<ip-address>]
                               [--dns-nameserver <dns-nameserver>]
                               [--host-route destination=<subnet>,gateway=<ip-address>]
                               [--service-type <service-type>]
                               [--tag <tag> | --no-tag]
                               name

创建一个 IPV4 指定地址范围的子网

[root@10e150e68e69 ~]# openstack subnet create --dns-nameserver 114.114.114.114 --dhcp --ip-version 4 --network f9184f28-c7c0-45cc-a1bb-17bb710cd17d --subnet-range 192.168.66.0/24 ops_test1
+-------------------+--------------------------------------+
| Field             | Value                                |
+-------------------+--------------------------------------+
| allocation_pools  | 192.168.66.2-192.168.66.254          |
| cidr              | 192.168.66.0/24                      |
| created_at        | 2018-08-10T08:22:21Z                 |
| description       |                                      |
| dns_nameservers   | 114.114.114.114                      |
| enable_dhcp       | True                                 |
| gateway_ip        | 192.168.66.1                         |
| host_routes       |                                      |
| id                | 85ab7188-86bb-4819-976b-d39d0c3812c7 |
| ip_version        | 4                                    |
| ipv6_address_mode | None                                 |
| ipv6_ra_mode      | None                                 |
| name              | ops_test1                            |
| network_id        | f9184f28-c7c0-45cc-a1bb-17bb710cd17d |
| project_id        | 72fc263931ec46529f18ed7450432fe7     |
| revision_number   | 0                                    |
| segment_id        | None                                 |
| service_types     |                                      |
| status            | ACTIVE                               |
| subnetpool_id     | None                                 |
| tags              |                                      |
| updated_at        | 2018-08-10T08:22:21Z                 |
+-------------------+--------------------------------------+

子网删除

[root@10e150e68e69 ~]# openstack subnet delete 85ab7188-86bb-4819-976b-d39d0c3812c7

验证

[root@10e150e68e69 ~]# openstack subnet list |grep 85ab7188-86bb-4819-976b-d39d0c3812c7

子网更新

usage: neutron subnet-update [-h] [--name NAME] [--description DESCRIPTION]
                          [--gateway GATEWAY_IP | --no-gateway]
                          [--allocation-pool start=IP_ADDR,end=IP_ADDR]
                          [--host-route destination=CIDR,nexthop=IP_ADDR]
                          [--dns-nameserver DNS_NAMESERVER]
                          [--disable-dhcp] [--enable-dhcp]
                          SUBNET

更新

[root@hb02-other-172e28e8e132 ~]# neutron subnet-update --allocation-pool start=192.168.100.2,end=192.168.100.200 4f173dec-ca44-49ce-8a91-110bc762b14a

验证

[root@hb02-other-172e28e8e132 ~]# neutron subnet-list
+--------------------------------------+-------+----------------------------------+------------------+------------------------------------------------------+
| id                                   | name  | tenant_id                        | cidr             | allocation_pools                                     |
+--------------------------------------+-------+----------------------------------+------------------+------------------------------------------------------+
| 4f173dec-ca44-49ce-8a91-110bc762b14a |       | 9e5b5032812940d0830fe674517d5f66 | 192.168.100.0/24 | {"start": "192.168.100.2", "end": "192.168.100.200"} |
| 7efd244e-4101-421d-8aa3-e5dc42834b9d | test1 | 9e5b5032812940d0830fe674517d5f66 | 192.168.101.0/24 | {"start": "192.168.101.2", "end": "192.168.101.254"} |
+--------------------------------------+-------+----------------------------------+------------------+------------------------------------------------------+

注解

当前版本的 openstack 命令执行子网更新存在问题

浮动 IP 创建

usage: openstack floating ip create [-h] [-f {json,shell,table,value,yaml}]
                                    [-c COLUMN] [--max-width <integer>]
                                    [--fit-width] [--print-empty] [--noindent]
                                    [--prefix PREFIX] [--subnet <subnet>]
                                    [--port <port>]
                                    [--floating-ip-address <ip-address>]
                                    [--fixed-ip-address <ip-address>]
                                    [--qos-policy <qos-policy>]
                                    [--description <description>]
                                    [--project <project>]
                                    [--project-domain <project-domain>]
                                    <network>

创建一个不指定其他参数的浮动 IP

[root@10e150e68e69 ~]# openstack floating ip create b79b7e9f-7894-4705-956e-3083f13f2993
+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| created_at          | 2018-08-13T06:15:49Z                 |
| description         |                                      |
| fixed_ip_address    | None                                 |
| floating_ip_address | 2.2.0.143                            |
| floating_network_id | b79b7e9f-7894-4705-956e-3083f13f2993 |
| id                  | 6f31c72e-36ef-4b9a-877d-eb2e01c3d094 |
| name                | 2.2.0.143                            |
| port_id             | None                                 |
| project_id          | 72fc263931ec46529f18ed7450432fe7     |
| qos_policy_id       | aaaaaaaa-aaaa-aaaa-aaaa-000000000001 |
| revision_number     | 1                                    |
| router_id           | None                                 |
| status              | DOWN                                 |
| subnet_id           | None                                 |
| updated_at          | 2018-08-13T06:15:49Z                 |
+---------------------+--------------------------------------+

验证

[root@10e150e68e69 ~]# openstack floating ip show 2.2.0.143
+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| created_at          | 2018-08-13T06:15:49Z                 |
| description         |                                      |
| fixed_ip_address    | None                                 |
| floating_ip_address | 2.2.0.143                            |
| floating_network_id | b79b7e9f-7894-4705-956e-3083f13f2993 |
| id                  | 6f31c72e-36ef-4b9a-877d-eb2e01c3d094 |
| name                | 2.2.0.143                            |
| port_id             | None                                 |
| project_id          | 72fc263931ec46529f18ed7450432fe7     |
| qos_policy_id       | aaaaaaaa-aaaa-aaaa-aaaa-000000000001 |
| revision_number     | 1                                    |
| router_id           | None                                 |
| status              | DOWN                                 |
| subnet_id           | None                                 |
| updated_at          | 2018-08-13T06:15:49Z                 |
+---------------------+--------------------------------------+

浮动 IP 关联端口

[root@10e150e68e69 ~]# openstack floating ip set --port 74d23bb6-4d6e-43a2-852c-cf1530333c8a 2.2.0.143

验证

[root@10e150e68e69 ~]# openstack floating ip show 2.2.0.143
+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| created_at          | 2018-08-13T06:15:49Z                 |
| description         |                                      |
| fixed_ip_address    | 12.12.12.96                          |
| floating_ip_address | 2.2.0.143                            |
| floating_network_id | b79b7e9f-7894-4705-956e-3083f13f2993 |
| id                  | 6f31c72e-36ef-4b9a-877d-eb2e01c3d094 |
| name                | 2.2.0.143                            |
| port_id             | 74d23bb6-4d6e-43a2-852c-cf1530333c8a |
| project_id          | 72fc263931ec46529f18ed7450432fe7     |
| qos_policy_id       | aaaaaaaa-aaaa-aaaa-aaaa-000000000001 |
| revision_number     | 3                                    |
| router_id           | 05cb7c0c-1f85-4a54-94da-750744f643d1 |
| status              | ACTIVE                               |
| subnet_id           | None                                 |
| updated_at          | 2018-08-13T06:27:49Z                 |
+---------------------+--------------------------------------+

浮动 IP 取消端口关联

[root@10e150e68e69 ~]# openstack floating ip unset --port 2.2.0.143

验证

[root@10e150e68e69 ~]# openstack floating ip show 2.2.0.143
+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| created_at          | 2018-08-13T06:15:49Z                 |
| description         |                                      |
| fixed_ip_address    | None                                 |
| floating_ip_address | 2.2.0.143                            |
| floating_network_id | b79b7e9f-7894-4705-956e-3083f13f2993 |
| id                  | 6f31c72e-36ef-4b9a-877d-eb2e01c3d094 |
| name                | 2.2.0.143                            |
| port_id             | None                                 |
| project_id          | 72fc263931ec46529f18ed7450432fe7     |
| qos_policy_id       | aaaaaaaa-aaaa-aaaa-aaaa-000000000001 |
| revision_number     | 6                                    |
| router_id           | None                                 |
| status              | DOWN                                 |
| subnet_id           | None                                 |
| updated_at          | 2018-08-13T06:34:13Z                 |
+---------------------+--------------------------------------+

浮动 IP 删除

[root@10e150e68e69 ~]# openstack floating ip delete 2.2.0.143

验证

[root@10e150e68e69 ~]# openstack floating ip list |grep '2.2.0.143'

路由创建

usage: openstack router create [-h] [-f {json,shell,table,value,yaml}]
                               [-c COLUMN] [--max-width <integer>]
                               [--fit-width] [--print-empty] [--noindent]
                               [--prefix PREFIX] [--enable | --disable]
                               [--distributed | --centralized]
                               [--ha | --no-ha] [--description <description>]
                               [--project <project>]
                               [--project-domain <project-domain>]
                               [--availability-zone-hint <availability-zone>]
                               [--tag <tag> | --no-tag]
                               <name>

创建一个路由

[root@10e150e68e69 ~]# openstack router create ops_test
+-------------------------+--------------------------------------+
| Field                   | Value                                |
+-------------------------+--------------------------------------+
| admin_state_up          | UP                                   |
| availability_zone_hints |                                      |
| availability_zones      |                                      |
| created_at              | 2018-08-10T08:43:00Z                 |
| description             |                                      |
| distributed             | True                                 |
| external_gateway_info   | None                                 |
| flavor_id               | None                                 |
| ha                      | False                                |
| id                      | 2fbfda60-74d9-49dc-8b75-c194431b6a3f |
| name                    | ops_test                             |
| project_id              | 72fc263931ec46529f18ed7450432fe7     |
| revision_number         | 1                                    |
| routes                  |                                      |
| status                  | ACTIVE                               |
| tags                    |                                      |
| updated_at              | 2018-08-10T08:43:00Z                 |
+-------------------------+--------------------------------------+

路由删除

[root@10e150e68e69 ~]# openstack router delete 2fbfda60-74d9-49dc-8b75-c194431b6a3f

验证

[root@10e150e68e69 ~]# openstack router list |grep 2fbfda60-74d9-49dc-8b75-c194431b6a3f

路由添加

usage: openstack network agent add router [-h] [--l3] <agent-id> <router>

路由添加到 Agent

[root@hn02-control-10e116e1e13 ~]# openstack network agent add router --l3 b0dc7298-092a-458f-bc74-aaffee2a34a2 06592146-86ed-4831-8822-4b9698ca7092

验证

[root@hn02-control-10e116e1e13 ~]# neutron router-list-on-l3-agent b0dc7298-092a-458f-bc74-aaffee2a34a2
+--------------------------------------+-----------------------------------------------------+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id                                   | name                                                | tenant_id                        | external_gateway_info                                                                                                                                                                                                                              |
+--------------------------------------+-----------------------------------------------------+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 06592146-86ed-4831-8822-4b9698ca7092 | default_router-b96b386f-2cea-4f9e-891f-0f7b9d19026c | d8e6d09230b64f08b8bb8be68a2b5b34 | {"network_id": "a857ac40-7e89-4eb3-be01-754dc4f3da82", "enable_snat": false, "qos_policy_id": "aaaaaaaa-aaaa-aaaa-aaaa-000000000001", "external_fixed_ips": [{"subnet_id": "5ef23aa8-0e3a-4fd9-a920-3e0624e90d4a", "ip_address": "10.116.3.11"}]}  |
+--------------------------------------+-----------------------------------------------------+------------

路由移除

usage: openstack network agent remove router [-h] [--l3] <agent-id> <router>

从 Agent 移除路由

[root@hn02-control-10e116e1e13 ~]# openstack network agent remove router --l3 b0dc7298-092a-458f-bc74-aaffee2a34a2 06592146-86ed-4831-8822-4b9698ca7092

验证

[root@hn02-control-10e116e1e13 ~]# neutron router-list-on-l3-agent b0dc7298-092a-458f-bc74-aaffee2a34a2
+--------------------------------------+-----------------------------------------------------+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id                                   | name                                                | tenant_id                        | external_gateway_info                                                                                                                                                                                                                              |
+--------------------------------------+-----------------------------------------------------+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                      |                                                     |                                  |                                                                                                                                                                                                                                                    |
+--------------------------------------+-----------------------------------------------------+---------------------

路由切换

手动切换主备 router,一个 router 有两个 keepalived 进程

# ps -ef|grep <router_id>

需要kill 掉父进程号不为 1 的 keepalived 进程,切记不能执行 kill -9 命令

# kill <pid>

验证

# ps -ef|grep 19237afb-65f7-4293-b476-3889aeb5b0e6
neutron   31974      1  0 01:42 ?        00:00:00 /usr/bin/python2 /bin/neutron-keepalived-state-change --router_id=19237afb-65f7-4293-b476-3889aeb5b0e6 --namespace=snat-19237afb-65f7-4293-b476-3889aeb5b0e6 --conf_dir=/var/lib/neutron/ha_confs/19237afb-65f7-4293-b476-3889aeb5b0e6 --monitor_interface=ha-906501dd-dd --monitor_cidr=169.254.0.2/24 --pid_file=/var/lib/neutron/external/pids/19237afb-65f7-4293-b476-3889aeb5b0e6.monitor.pid --state_path=/var/lib/neutron --user=995 --group=993 --AGENT-root_helper=sudo neutron-rootwrap /etc/neutron/rootwrap.conf --AGENT-root_helper_daemon=
root      33813      1  0 01:43 ?        00:00:01 keepalived -P -f /var/lib/neutron/ha_confs/19237afb-65f7-4293-b476-3889aeb5b0e6/keepalived.conf -p /var/lib/neutron/ha_confs/19237afb-65f7-4293-b476-3889aeb5b0e6.pid -r /var/lib/neutron/ha_confs/19237afb-65f7-4293-b476-3889aeb5b0e6.pid-vrrp -D
root      33814  33813  0 01:43 ?        00:00:04 keepalived -P -f /var/lib/neutron/ha_confs/19237afb-65f7-4293-b476-3889aeb5b0e6/keepalived.conf -p /var/lib/neutron/ha_confs/19237afb-65f7-4293-b476-3889aeb5b0e6.pid -r /var/lib/neutron/ha_confs/19237afb-65f7-4293-b476-3889aeb5b0e6.pid-vrrp -D
# kill 33814

路由移除、路由添加和路由切换用于网络结点故障的场景中,当网络结点故障时,先添加路由到新的 l3-agent,再移除掉原 l3-agent 上面的路由。

路由查找

根据 agent 查找路由

usage: openstack router list [-h] [--agent <agent-id>]

usage: neutron router-list-on-l3-agent <L3_AGENT_ID>

验证

[root@ln01-control-10e148e44e13 ~]# openstack network agent list --agent-type l3
+--------------------------------------+------------+---------------------------+-------------------+-------+-------+------------------+
| ID                                   | Agent Type | Host                      | Availability Zone | Alive | State | Binary           |
+--------------------------------------+------------+---------------------------+-------------------+-------+-------+------------------+
| aa354271-fc9f-47d1-8571-a410c18232bf | L3 agent   | ln01-compute-10e148e44e30 | nova              | :-)   | UP    | neutron-l3-agent |
| 08bacce4-47a8-4688-a1f6-6f9ddc06df89 | L3 agent   | ln01-compute-10e148e44e31 | nova              | :-)   | UP    | neutron-l3-agent |
| 1b059b98-8e22-4c9a-927a-461443dd8c86 | L3 agent   | ln01-network-10e148e44e28 | nova              | :-)   | UP    | neutron-l3-agent |
+--------------------------------------+------------+---------------------------+-------------------+-------+-------+------------------+

[root@ln01-control-10e148e44e13 ~]# openstack router list --agent 1b059b98-8e22-4c9a-927a-461443dd8c86
+--------------------------------------+-----------------------------------------------------+--------+-------+-------------+------+----------------------------------+
| ID                                   | Name                                                | Status | State | Distributed | HA   | Project                          |
+--------------------------------------+-----------------------------------------------------+--------+-------+-------------+------+----------------------------------+
| 12017df3-e94d-4561-acc8-c9db217fe8cd | default_router-605e978b-e60e-4424-b238-7b0124af12d5 | ACTIVE | UP    | True        | True | 6407e103dd994f9cbcfe8f59cb33b94b |
| ca205f8d-bf2c-4c7f-a537-623687bddcb9 | default_router-76ef4dc3-239f-46a5-9302-0048c8287cad | ACTIVE | UP    | True        | True | 840167d74bce41039744dd488ff0045e |
| f04cb975-1b93-483e-b867-5c970e8b0976 | default_router-28988d0d-88c8-47b7-9533-c0b43b110e65 | ACTIVE | UP    | True        | True | 867a54a166564a548bc8616d2e9e29c4 |
+--------------------------------------+-----------------------------------------------------+--------+-------+-------------+------+----------------------------------+

[root@ln01-control-10e148e44e13 ~]# neutron router-list-on-l3-agent 1b059b98-8e22-4c9a-927a-461443dd8c86
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
+--------------------------------------+-----------------------------------------------------+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id                                   | name                                                | tenant_id                        | external_gateway_info                                                                                                                                                                                                                               |
+--------------------------------------+-----------------------------------------------------+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 12017df3-e94d-4561-acc8-c9db217fe8cd | default_router-605e978b-e60e-4424-b238-7b0124af12d5 | 6407e103dd994f9cbcfe8f59cb33b94b | {"network_id": "990dd1ae-b90d-4b44-8b5f-fc78b5ad48a1", "enable_snat": false, "qos_policy_id": "aaaaaaaa-aaaa-aaaa-aaaa-000000000001", "external_fixed_ips": [{"subnet_id": "3371548f-28b4-41c0-8169-fa8056cdda78", "ip_address": "10.148.65.10"}]}  |
| ca205f8d-bf2c-4c7f-a537-623687bddcb9 | default_router-76ef4dc3-239f-46a5-9302-0048c8287cad | 840167d74bce41039744dd488ff0045e | {"network_id": "990dd1ae-b90d-4b44-8b5f-fc78b5ad48a1", "enable_snat": false, "qos_policy_id": "aaaaaaaa-aaaa-aaaa-aaaa-000000000001", "external_fixed_ips": [{"subnet_id": "169b5845-32e7-4c05-aca7-1331bf670cb3", "ip_address": "100.65.128.18"}]} |
| f04cb975-1b93-483e-b867-5c970e8b0976 | default_router-28988d0d-88c8-47b7-9533-c0b43b110e65 | 867a54a166564a548bc8616d2e9e29c4 | {"network_id": "990dd1ae-b90d-4b44-8b5f-fc78b5ad48a1", "enable_snat": false, "qos_policy_id": "aaaaaaaa-aaaa-aaaa-aaaa-000000000001", "external_fixed_ips": [{"subnet_id": "3371548f-28b4-41c0-8169-fa8056cdda78", "ip_address": "10.148.65.15"}]}  |
+--------------------------------------+-----------------------------------------------------+------------------

根据 port 查找路由

usage: openstack port find router [-h] <port>

验证

[root@hb02-other-172e28e8e132 ~]# openstack port find router dd8531b7-6015-4ca9-ad1c-4812f39f8fc7
7fb30499-3088-416b-ac1d-c9f6cf1f491a

根据路由查 port

usage: openstack port list [--router <router>]

usage: neutron router-port-list <ROUTER>

验证

[root@hb02-other-172e28e8e132 ~]# openstack port list --router 1602f723-8930-401e-9685-12bd17eef6cf
+--------------------------------------+-------------------------------------------------+-------------------+-------------------------------------------------------------------------------+--------+
| ID                                   | Name                                            | MAC Address       | Fixed IP Addresses                                                            | Status |
+--------------------------------------+-------------------------------------------------+-------------------+-------------------------------------------------------------------------------+--------+
| c976f974-eafe-4310-9d55-0a338fff31d4 | HA port tenant 9e5b5032812940d0830fe674517d5f66 | fa:16:3e:67:ce:93 | ip_address='169.254.192.4', subnet_id='e5af4553-27d1-4c90-8eba-3d54b29fce9e'  | DOWN   |
| f7d946a5-1cdf-4464-97a8-1c02c3377cb3 | HA port tenant 9e5b5032812940d0830fe674517d5f66 | fa:16:3e:7a:70:72 | ip_address='169.254.192.23', subnet_id='e5af4553-27d1-4c90-8eba-3d54b29fce9e' | DOWN   |
+--------------------------------------+-------------------------------------------------+-------------------+-------------------------------------------------------------------------------+--------+

[root@hb02-other-172e28e8e132 ~]# neutron router-port-list 1602f723-8930-401e-9685-12bd17eef6cf
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
+--------------------------------------+-------------------------------------------------+-----------+-------------------+---------------------------------------------------------------------------------------+
| id                                   | name                                            | tenant_id | mac_address       | fixed_ips                                                                             |
+--------------------------------------+-------------------------------------------------+-----------+-------------------+---------------------------------------------------------------------------------------+
| c976f974-eafe-4310-9d55-0a338fff31d4 | HA port tenant 9e5b5032812940d0830fe674517d5f66 |           | fa:16:3e:67:ce:93 | {"subnet_id": "e5af4553-27d1-4c90-8eba-3d54b29fce9e", "ip_address": "169.254.192.4"}  |
| f7d946a5-1cdf-4464-97a8-1c02c3377cb3 | HA port tenant 9e5b5032812940d0830fe674517d5f66 |           | fa:16:3e:7a:70:72 | {"subnet_id": "e5af4553-27d1-4c90-8eba-3d54b29fce9e", "ip_address": "169.254.192.23"} |
+--------------------------------------+-------------------------------------------------+-----------+--------

路由编辑

usage: openstack router set [-h] [--name <name>] [--description <description>]
                            [--enable | --disable]
                            [--distributed | --centralized]
                            [--route destination=<subnet>,gateway=<ip-address>]
                            [--no-route] [--ha | --no-ha]
                            [--external-gateway <network>]
                            [--fixed-ip subnet=<subnet>,ip-address=<ip-address>]
                            [--enable-snat | --disable-snat]
                            [--qos-policy <qos-policy> | --no-qos-policy]
                            [--tag <tag>] [--no-tag]
                            <router>

路由连接外网

[root@10e150e68e69 ~]# openstack router set --external-gateway ext-net ops_test

验证

[root@10e150e68e69 ~]# openstack router show ops_test
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field                   | Value                                                                                                                                                                                                                                           |
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up          | UP                                                                                                                                                                                                                                              |
| availability_zone_hints |                                                                                                                                                                                                                                                 |
| availability_zones      | nova                                                                                                                                                                                                                                            |
| created_at              | 2018-08-14T07:33:38Z                                                                                                                                                                                                                            |
| description             |                                                                                                                                                                                                                                                 |
| distributed             | True                                                                                                                                                                                                                                            |
| external_gateway_info   | {"network_id": "b79b7e9f-7894-4705-956e-3083f13f2993", "enable_snat": false, "qos_policy_id": "aaaaaaaa-aaaa-aaaa-aaaa-000000000001", "external_fixed_ips": [{"subnet_id": "8ba4676f-7656-4a6d-84af-8d8478dd4d0b", "ip_address": "2.2.0.149"}]} |
| flavor_id               | None                                                                                                                                                                                                                                            |
| ha                      | False                                                                                                                                                                                                                                           |
| id                      | b4ba4c9f-92ce-4ed2-8570-1e7963b201ed                                                                                                                                                                                                            |
| interfaces_info         | []                                                                                                                                                                                                                                              |
| name                    | ops_test                                                                                                                                                                                                                                        |
| project_id              | 72fc263931ec46529f18ed7450432fe7                                                                                                                                                                                                                |
| revision_number         | 4                                                                                                                                                                                                                                               |
| routes                  |                                                                                                                                                                                                                                                 |
| status                  | ACTIVE                                                                                                                                                                                                                                          |
| tags                    |                                                                                                                                                                                                                                                 |
| updated_at              | 2018-08-14T07:33:52Z                                                                                                                                                                                                                            |
+-------------------------+---------------------------------------------------------------------------------------

路由清除网关

[root@10e129e169e140 ~]# openstack router unset --external-gateway c57b4dc3-5a7c-4c06-a4ad-f47b57624c09

验证

[root@10e129e169e140 ~]# openstack router show c57b4dc3-5a7c-4c06-a4ad-f47b57624c09
+-------------------------+--------------------------------------+
| Field                   | Value                                |
+-------------------------+--------------------------------------+
| admin_state_up          | UP                                   |
| availability_zone_hints |                                      |
| availability_zones      | nova                                 |
| created_at              | 2018-08-21T07:52:42Z                 |
| description             |                                      |
| distributed             | True                                 |
| external_gateway_info   | None                                 |
| flavor_id               | None                                 |
| ha                      | False                                |
| id                      | c57b4dc3-5a7c-4c06-a4ad-f47b57624c09 |
| interfaces_info         | []                                   |
| name                    | ops_test                             |
| project_id              | 5e2214ea913a49babd5a4aab26e56ff8     |
| revision_number         | 11                                   |
| routes                  |                                      |
| status                  | ACTIVE                               |
| tags                    |                                      |
| updated_at              | 2018-08-21T08:16:53Z                 |
+-------------------------+--------------------------------------+

路由添加子网

[root@10e150e68e69 ~]# openstack router add subnet 05cb7c0c-1f85-4a54-94da-750744f643d1 65a7a74e-9277-4bcc-b720-20bfe17c376e

验证

[root@10e150e68e69 ~]# openstack router show 05cb7c0c-1f85-4a54-94da-750744f643d1
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field                   | Value                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up          | UP                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| availability_zone_hints |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| availability_zones      | nova                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| created_at              | 2018-07-05T07:40:12Z                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| description             |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| distributed             | True                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| external_gateway_info   | {"network_id": "b79b7e9f-7894-4705-956e-3083f13f2993", "enable_snat": false, "external_fixed_ips": [{"subnet_id": "f839bc50-691f-4cc4-b73e-eed89eecd7dd", "ip_address": "36.111.164.41"}]}                                                                                                                                                                                                                                                                                                                                                                    |
| flavor_id               | None                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| ha                      | False                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| id                      | 05cb7c0c-1f85-4a54-94da-750744f643d1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| interfaces_info         | [{"subnet_id": "65a7a74e-9277-4bcc-b720-20bfe17c376e", "ip_address": "192.168.66.1", "port_id": "037cc2ca-54ea-4aef-8d02-6262d0b8cfd3"}, {"subnet_id": "65a7a74e-9277-4bcc-b720-20bfe17c376e", "ip_address": "192.168.66.6", "port_id": "7b0d1664-4cc7-4268-870b-c0c4d6ece196"}, {"subnet_id": "a17553e1-6811-411e-b997-da7771f4b0ac", "ip_address": "12.12.12.1", "port_id": "c462ec33-c527-4a82-b345-d319d6f00975"}, {"subnet_id": "a17553e1-6811-411e-b997-da7771f4b0ac", "ip_address": "12.12.12.13", "port_id": "f27f0ca5-f746-45b2-a08c-345e38b87d04"}] |
| name                    | wwz-router2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| project_id              | 1a74867b94b8464ea3573dbcb38875ee                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| revision_number         | 7                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| routes                  |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| status                  | ACTIVE                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| tags                    |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| updated_at              | 2018-08-14T07:23:31Z                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
+-------------------------+--------------------------------------------------------------------------

路由移除子网

[root@10e150e68e69 ~]# openstack router remove subnet 05cb7c0c-1f85-4a54-94da-750744f643d1 65a7a74e-9277-4bcc-b720-20bfe17c376e

验证

[root@10e150e68e69 ~]# openstack router show 05cb7c0c-1f85-4a54-94da-750744f643d1
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field                   | Value                                                                                                                                                                                                                                                                         |
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up          | UP                                                                                                                                                                                                                                                                            |
| availability_zone_hints |                                                                                                                                                                                                                                                                               |
| availability_zones      | nova                                                                                                                                                                                                                                                                          |
| created_at              | 2018-07-05T07:40:12Z                                                                                                                                                                                                                                                          |
| description             |                                                                                                                                                                                                                                                                               |
| distributed             | True                                                                                                                                                                                                                                                                          |
| external_gateway_info   | {"network_id": "b79b7e9f-7894-4705-956e-3083f13f2993", "enable_snat": false, "external_fixed_ips": [{"subnet_id": "f839bc50-691f-4cc4-b73e-eed89eecd7dd", "ip_address": "36.111.164.41"}]}                                                                                    |
| flavor_id               | None                                                                                                                                                                                                                                                                          |
| ha                      | False                                                                                                                                                                                                                                                                         |
| id                      | 05cb7c0c-1f85-4a54-94da-750744f643d1                                                                                                                                                                                                                                          |
| interfaces_info         | [{"subnet_id": "a17553e1-6811-411e-b997-da7771f4b0ac", "ip_address": "12.12.12.1", "port_id": "c462ec33-c527-4a82-b345-d319d6f00975"}, {"subnet_id": "a17553e1-6811-411e-b997-da7771f4b0ac", "ip_address": "12.12.12.13", "port_id": "f27f0ca5-f746-45b2-a08c-345e38b87d04"}] |
| name                    | wwz-router2                                                                                                                                                                                                                                                                   |
| project_id              | 1a74867b94b8464ea3573dbcb38875ee                                                                                                                                                                                                                                              |
| revision_number         | 9                                                                                                                                                                                                                                                                             |
| routes                  |                                                                                                                                                                                                                                                                               |
| status                  | ACTIVE                                                                                                                                                                                                                                                                        |
| tags                    |                                                                                                                                                                                                                                                                               |
| updated_at              | 2018-08-14T09:57:32Z                                                                                                                                                                                                                                                          |
+-------------------------+------------------------------------------------------------------------------------------------

qos 创建

usage: openstack network qos rule create [-h]
                                         [-f {json,shell,table,value,yaml}]
                                         [-c COLUMN] [--max-width <integer>]
                                         [--fit-width] [--print-empty]
                                         [--noindent] [--prefix PREFIX]
                                         [--type <type>]
                                         [--max-kbps <max-kbps>]
                                         [--max-burst-kbits <max-burst-kbits>]
                                         [--dscp-mark <dscp-mark>]
                                         [--min-kbps <min-kbps>]
                                         [--ingress | --egress]
                                         <qos-policy>

创建出口带宽限速规则

[root@10e150e68e69 open_scr]# openstack network qos rule create --type bandwidth-limit --max-kbps 15000  --max-burst-kbits 0 --egress 72dea9cb-c12f-41ee-a0d4-9170af5dafbc
+----------------+--------------------------------------+
| Field          | Value                                |
+----------------+--------------------------------------+
| direction      | egress                               |
| id             | 036ad1c7-54f5-4d33-91d0-a80c4b3d1146 |
| max_burst_kbps | 0                                    |
| max_kbps       | 15000                                |
| name           | None                                 |
| project_id     |                                      |
+----------------+--------------------------------------+

qos 规则修改

修改带宽限速规则

usage: openstack network qos rule set [--max-kbps <max-kbps>]
                                      [--max-burst-kbits <max-burst-kbits>]
                                      [--min-kbps <min-kbps>]
                                      [--ingress | --egress]
                                      <qos-policy> <rule-id>

验证

# openstack network qos rule set --max-kbps 12000 --egress 72dea9cb-c12f-41ee-a0d4-9170af5dafbc 036ad1c7-54f5-4d33-91d0-a80c4b3d1146
# openstack network qos rule show 72dea9cb-c12f-41ee-a0d4-9170af5dafbc 036ad1c7-54f5-4d33-91d0-a80c4b3d1146
+----------------+--------------------------------------+
| Field          | Value                                |
+----------------+--------------------------------------+
| direction      | egress                               |
| id             | 036ad1c7-54f5-4d33-91d0-a80c4b3d1146 |
| max_burst_kbps | 0                                    |
| max_kbps       | 12000                                |
| name           | None                                 |
| project_id     |                                      |
+----------------+--------------------------------------+

端口创建

usage: openstack port create [-h] [-f {json,shell,table,value,yaml}]
                             [-c COLUMN] [--max-width <integer>] [--fit-width]
                             [--print-empty] [--noindent] [--prefix PREFIX]
                             --network <network> [--description <description>]
                             [--device <device-id>]
                             [--mac-address <mac-address>]
                             [--device-owner <device-owner>]
                             [--vnic-type <vnic-type>] [--host <host-id>]
                             [--dns-name dns-name]
                             [--fixed-ip subnet=<subnet>,ip-address=<ip-address> | --no-fixed-ip]
                             [--binding-profile <binding-profile>]
                             [--enable | --disable] [--project <project>]
                             [--project-domain <project-domain>]
                             [--security-group <security-group> | --no-security-group]
                             [--qos-policy <qos-policy>]
                             [--enable-port-security | --disable-port-security]
                             [--allowed-address ip-address=<ip-address>[,mac-address=<mac-address>]]
                             [--tag <tag> | --no-tag]
                             <name>

创建一个指定 IP 的端口

[root@10e150e68e69 ~]# openstack port create --network 94850914-6f53-4af4-a263-02ead62277b8 --fixed-ip subnet=a17553e1-6811-411e-b997-da7771f4b0ac,ip-address=12.12.12.66 ops_test
+-----------------------+----------------------------------------------------------------------------+
| Field                 | Value                                                                      |
+-----------------------+----------------------------------------------------------------------------+
| admin_state_up        | UP                                                                         |
| allowed_address_pairs |                                                                            |
| binding_host_id       |                                                                            |
| binding_profile       |                                                                            |
| binding_vif_details   |                                                                            |
| binding_vif_type      | unbound                                                                    |
| binding_vnic_type     | normal                                                                     |
| created_at            | 2018-08-10T09:00:04Z                                                       |
| data_plane_status     | None                                                                       |
| description           |                                                                            |
| device_id             |                                                                            |
| device_owner          |                                                                            |
| dns_assignment        | None                                                                       |
| dns_name              | None                                                                       |
| extra_dhcp_opts       |                                                                            |
| fixed_ips             | ip_address='12.12.12.66', subnet_id='a17553e1-6811-411e-b997-da7771f4b0ac' |
| id                    | b70c0c87-b09d-4535-8e9f-bb732322ab74                                       |
| ip_address            | None                                                                       |
| mac_address           | fa:16:3e:f4:e8:9a                                                          |
| name                  | ops_test                                                                   |
| network_id            | 94850914-6f53-4af4-a263-02ead62277b8                                       |
| option_name           | None                                                                       |
| option_value          | None                                                                       |
| port_security_enabled | True                                                                       |
| project_id            | 72fc263931ec46529f18ed7450432fe7                                           |
| qos_policy_id         | None                                                                       |
| revision_number       | 6                                                                          |
| security_group_ids    | 67e2aef8-7fcc-4a44-8318-c34c89a9f025                                       |
| status                | DOWN                                                                       |
| subnet_id             | None                                                                       |
| tags                  |                                                                            |
| trunk_details         | None                                                                       |
| updated_at            | 2018-08-10T09:00:04Z                                                       |
+-----------------------+----------------------------------------------------------------------------+

验证

[root@10e150e68e69 ~]# openstack port list --network 94850914-6f53-4af4-a263-02ead62277b8
+--------------------------------------+----------+-------------------+----------------------------------------------------------------------------+--------+
| ID                                   | Name     | MAC Address       | Fixed IP Addresses                                                         | Status |
+--------------------------------------+----------+-------------------+----------------------------------------------------------------------------+--------+
| 2c809e68-b1b3-4a4f-a245-9922b24e3e51 |          | fa:16:3e:af:9a:33 | ip_address='12.12.12.2', subnet_id='a17553e1-6811-411e-b997-da7771f4b0ac'  | ACTIVE |
| 5a07faea-2966-44b4-8b19-66d0f88080f4 |          | fa:16:3e:51:5c:cf | ip_address='12.12.12.5', subnet_id='a17553e1-6811-411e-b997-da7771f4b0ac'  | ACTIVE |
| b70c0c87-b09d-4535-8e9f-bb732322ab74 | ops_test | fa:16:3e:f4:e8:9a | ip_address='12.12.12.66', subnet_id='a17553e1-6811-411e-b997-da7771f4b0ac' | DOWN   |
| c462ec33-c527-4a82-b345-d319d6f00975 |          | fa:16:3e:0f:a3:dc | ip_address='12.12.12.1', subnet_id='a17553e1-6811-411e-b997-da7771f4b0ac'  | ACTIVE |
| c749e651-cc55-4f1e-ac6c-16ca955b7bc3 |          | fa:16:3e:2c:4b:5e | ip_address='12.12.12.11', subnet_id='a17553e1-6811-411e-b997-da7771f4b0ac' | ACTIVE |
| dcdd58cd-7a61-42c8-a4e6-c8a697ed1986 |          | fa:16:3e:6b:c7:b7 | ip_address='12.12.12.3', subnet_id='a17553e1-6811-411e-b997-da7771f4b0ac'  | ACTIVE |
| f27f0ca5-f746-45b2-a08c-345e38b87d04 |          | fa:16:3e:ad:ee:63 | ip_address='12.12.12.13', subnet_id='a17553e1-6811-411e-b997-da7771f4b0ac' | ACTIVE |
+--------------------------------------+----------+-------------------+------------------------------------------

创建一个不指定 IP 的端口

[root@10e150e68e69 ~]# openstack port create ops_port2 --network 94850914-6f53-4af4-a263-02ead62277b8
+-----------------------+---------------------------------------------------------------------------+
| Field                 | Value                                                                     |
+-----------------------+---------------------------------------------------------------------------+
| admin_state_up        | UP                                                                        |
| allowed_address_pairs |                                                                           |
| binding_host_id       |                                                                           |
| binding_profile       |                                                                           |
| binding_vif_details   |                                                                           |
| binding_vif_type      | unbound                                                                   |
| binding_vnic_type     | normal                                                                    |
| created_at            | 2018-08-14T08:29:37Z                                                      |
| data_plane_status     | None                                                                      |
| description           |                                                                           |
| device_id             |                                                                           |
| device_owner          |                                                                           |
| dns_assignment        | None                                                                      |
| dns_name              | None                                                                      |
| extra_dhcp_opts       |                                                                           |
| fixed_ips             | ip_address='12.12.12.8', subnet_id='a17553e1-6811-411e-b997-da7771f4b0ac' |
| id                    | c830488f-3614-4634-a394-21756617bb6a                                      |
| ip_address            | None                                                                      |
| mac_address           | fa:16:3e:a9:38:2e                                                         |
| name                  | ops_port2                                                                 |
| network_id            | 94850914-6f53-4af4-a263-02ead62277b8                                      |
| option_name           | None                                                                      |
| option_value          | None                                                                      |
| port_security_enabled | True                                                                      |
| project_id            | 72fc263931ec46529f18ed7450432fe7                                          |
| qos_policy_id         | None                                                                      |
| revision_number       | 6                                                                         |
| security_group_ids    | 67e2aef8-7fcc-4a44-8318-c34c89a9f025                                      |
| status                | DOWN                                                                      |
| subnet_id             | None                                                                      |
| tags                  |                                                                           |
| trunk_details         | None                                                                      |
| updated_at            | 2018-08-14T08:29:37Z                                                      |
+-----------------------+---------------------------------------------------------------------------+

端口编辑

usage: openstack port set [-h] [--description <description>]
                          [--device <device-id>] [--mac-address <mac-address>]
                          [--device-owner <device-owner>]
                          [--vnic-type <vnic-type>] [--host <host-id>]
                          [--dns-name dns-name] [--enable | --disable]
                          [--name <name>]
                          [--fixed-ip subnet=<subnet>,ip-address=<ip-address>]
                          [--no-fixed-ip]
                          [--binding-profile <binding-profile>]
                          [--no-binding-profile] [--qos-policy <qos-policy>]
                          [--security-group <security-group>]
                          [--no-security-group]
                          [--enable-port-security | --disable-port-security]
                          [--allowed-address ip-address=<ip-address>[,mac-address=<mac-address>]]
                          [--no-allowed-address]
                          [--data-plane-status <status>] [--tag <tag>]
                          [--no-tag]
                          <port>

端口绑定一个 qos 策略

[root@10e150e68e69 open_scr]# openstack port set --qos-policy 72dea9cb-c12f-41ee-a0d4-9170af5dafbc 86a5368f-9f69-4ceb-b301-072b2ea823ee

验证

[root@10e150e68e69 open_scr]# openstack port show 86a5368f-9f69-4ceb-b301-072b2ea823ee
+-----------------------+----------------------------------------------------------------------------+
| Field                 | Value                                                                      |
+-----------------------+----------------------------------------------------------------------------+
| admin_state_up        | UP                                                                         |
| allowed_address_pairs |                                                                            |
| binding_host_id       |                                                                            |
| binding_profile       |                                                                            |
| binding_vif_details   |                                                                            |
| binding_vif_type      | unbound                                                                    |
| binding_vnic_type     | normal                                                                     |
| created_at            | 2018-08-21T09:09:00Z                                                       |
| data_plane_status     | None                                                                       |
| description           |                                                                            |
| device_id             |                                                                            |
| device_owner          |                                                                            |
| dns_assignment        | None                                                                       |
| dns_name              | None                                                                       |
| extra_dhcp_opts       |                                                                            |
| fixed_ips             | ip_address='12.12.12.66', subnet_id='a17553e1-6811-411e-b997-da7771f4b0ac' |
| id                    | 86a5368f-9f69-4ceb-b301-072b2ea823ee                                       |
| ip_address            | None                                                                       |
| mac_address           | fa:16:3e:74:65:6a                                                          |
| name                  | ops_test                                                                   |
| network_id            | 94850914-6f53-4af4-a263-02ead62277b8                                       |
| option_name           | None                                                                       |
| option_value          | None                                                                       |
| port_security_enabled | True                                                                       |
| project_id            | 72fc263931ec46529f18ed7450432fe7                                           |
| qos_policy_id         | 72dea9cb-c12f-41ee-a0d4-9170af5dafbc                                       |
| revision_number       | 8                                                                          |
| security_group_ids    | 67e2aef8-7fcc-4a44-8318-c34c89a9f025                                       |
| status                | DOWN                                                                       |
| subnet_id             | None                                                                       |
| tags                  |                                                                            |
| trunk_details         | None                                                                       |
| updated_at            | 2018-08-21T09:10:08Z                                                       |
+-----------------------+----------------------------------------------------------------------------+

查看租户网络配额

usage: neutron quota-show [-h] [-f {json,shell,table,value,yaml}] [-c COLUMN]
                       [--max-width <integer>] [--fit-width]
                       [--print-empty] [--noindent] [--prefix PREFIX]
                       [--tenant-id tenant-id]

查看配额

[root@hb02-other-172e28e8e132 ~]# neutron quota-show --tenant-id 9e5b5032812940d0830fe674517d5f66
+---------------------+-------+
| Field               | Value |
+---------------------+-------+
| firewall_group      | -1    |
| firewall_policy     | 10    |
| firewall_rule       | 100   |
| floatingip          | 100   |
| network             | 5     |
| port                | 600   |
| rbac_policy         | 10    |
| router              | 5     |
| security_group      | 100   |
| security_group_rule | 800   |
| subnet              | 100   |
| subnetpool          | -1    |
+---------------------+-------+

更新租户网络配额

                         [-c COLUMN] [--max-width <integer>] [--fit-width]
                         [--print-empty] [--noindent] [--prefix PREFIX]
                         [--tenant-id tenant-id] [--network networks]
                         [--subnet subnets] [--port ports]
                         [--router routers] [--floatingip floatingips]
                         [--security-group security_groups]
                         [--security-group-rule security_group_rules]
                         [--vip vips] [--pool pools] [--member members]
                         [--health-monitor health_monitors]
                         [--loadbalancer loadbalancers]
                         [--listener listeners]
                         [--rbac-policy rbac_policies]

更新 port 数量

[root@hb02-other-172e28e8e132 ~]# neutron quota-update  --tenant-id 9e5b5032812940d0830fe674517d5f66 --port 1000
+---------------------+-------+
| Field               | Value |
+---------------------+-------+
| firewall_group      | -1    |
| firewall_policy     | 10    |
| firewall_rule       | 100   |
| floatingip          | 100   |
| network             | 5     |
| port                | 1000  |
| rbac_policy         | 10    |
| router              | 5     |
| security_group      | 100   |
| security_group_rule | 800   |
| subnet              | 100   |
| subnetpool          | -1    |
+---------------------+-------+

安全组创建

usage: openstack security group create [-h] [-f {json,shell,table,value,yaml}]
                                       [-c COLUMN] [--max-width <integer>]
                                       [--fit-width] [--print-empty]
                                       [--noindent] [--prefix PREFIX]
                                       [--description <description>]
                                       [--project <project>]
                                       [--project-domain <project-domain>]
                                       <name>
[root@10e150e68e69 ~]# openstack security group create --description ops_test ops_test
+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field           | Value                                                                                                                                                                                                          |
+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| created_at      | 2018-08-13T07:38:42Z                                                                                                                                                                                           |
| description     | ops_test                                                                                                                                                                                                       |
| id              | 142c190e-2211-447f-9a20-fe86d5d40b60                                                                                                                                                                           |
| name            | ops_test                                                                                                                                                                                                       |
| project_id      | 72fc263931ec46529f18ed7450432fe7                                                                                                                                                                               |
| revision_number | 2                                                                                                                                                                                                              |
| rules           | created_at='2018-08-13T07:38:42Z', direction='ingress', ethertype='IPv4', id='22f55388-d654-4ef4-a04d-874df7757244', remote_group_id='142c190e-2211-447f-9a20-fe86d5d40b60', updated_at='2018-08-13T07:38:42Z' |
|                 | created_at='2018-08-13T07:38:42Z', direction='egress', ethertype='IPv4', id='c60f7349-4306-44ad-bced-5890131a0341', updated_at='2018-08-13T07:38:42Z'                                                          |
| updated_at      | 2018-08-13T07:38:42Z                                                                                                                                                                                           |
+-----------------+---------------------------------------------------------------------------------------------------

安全组规则创建

为指定网段创建 TCP 端口入口规则

[root@10e150e68e69 ~]# openstack security group rule create --protocol tcp --remote-ip 192.168.1.0/24 --ingress --dst-port 80 ops_test
+-------------------+--------------------------------------+
| Field             | Value                                |
+-------------------+--------------------------------------+
| created_at        | 2018-08-13T07:43:46Z                 |
| description       |                                      |
| direction         | ingress                              |
| ether_type        | IPv4                                 |
| id                | 96c58083-20db-4e2b-abc7-cd90c905f170 |
| name              | None                                 |
| port_range_max    | 80                                   |
| port_range_min    | 80                                   |
| project_id        | 72fc263931ec46529f18ed7450432fe7     |
| protocol          | tcp                                  |
| remote_group_id   | None                                 |
| remote_ip_prefix  | 192.168.1.0/24                       |
| revision_number   | 0                                    |
| security_group_id | 142c190e-2211-447f-9a20-fe86d5d40b60 |
| updated_at        | 2018-08-13T07:43:46Z                 |
+-------------------+--------------------------------------+

验证

[root@10e150e68e69 ~]# openstack security group rule list ops_test
+--------------------------------------+-------------+----------------+------------+--------------------------------------+
| ID                                   | IP Protocol | IP Range       | Port Range | Remote Security Group                |
+--------------------------------------+-------------+----------------+------------+--------------------------------------+
| 22f55388-d654-4ef4-a04d-874df7757244 | None        | None           |            | 142c190e-2211-447f-9a20-fe86d5d40b60 |
| 96c58083-20db-4e2b-abc7-cd90c905f170 | tcp         | 192.168.1.0/24 | 80:80      | None                                 |
| c60f7349-4306-44ad-bced-5890131a0341 | None        | None           |            | None                                 |
+--------------------------------------+-------------+----------------+------------+-------------------------------

创建 icmp 入口规则

[root@10e150e68e69 ~]# openstack security group rule create --protocol icmp --ingress  ops_test
+-------------------+--------------------------------------+
| Field             | Value                                |
+-------------------+--------------------------------------+
| created_at        | 2018-08-14T03:18:30Z                 |
| description       |                                      |
| direction         | ingress                              |
| ether_type        | IPv4                                 |
| id                | 63ec49aa-ac5f-4759-804b-f6f7054eb43e |
| name              | None                                 |
| port_range_max    | None                                 |
| port_range_min    | None                                 |
| project_id        | 72fc263931ec46529f18ed7450432fe7     |
| protocol          | icmp                                 |
| remote_group_id   | None                                 |
| remote_ip_prefix  | 0.0.0.0/0                            |
| revision_number   | 0                                    |
| security_group_id | 142c190e-2211-447f-9a20-fe86d5d40b60 |
| updated_at        | 2018-08-14T03:18:30Z                 |
+-------------------+--------------------------------------+

为指定 project 创建 udp 协议端口范围的入口规则

[root@10e150e68e69 ~]# openstack security group rule create --protocol udp --remote-ip 192.168.1.0/24 --ingress --dst-port 123:125 --project 1a74867b94b8464ea3573dbcb38875ee ops_test
+-------------------+--------------------------------------+
| Field             | Value                                |
+-------------------+--------------------------------------+
| created_at        | 2018-08-14T02:59:38Z                 |
| description       |                                      |
| direction         | ingress                              |
| ether_type        | IPv4                                 |
| id                | dbfee98e-fd37-44e7-a49a-af7ecbdd4173 |
| name              | None                                 |
| port_range_max    | 125                                  |
| port_range_min    | 123                                  |
| project_id        | 1a74867b94b8464ea3573dbcb38875ee     |
| protocol          | udp                                  |
| remote_group_id   | None                                 |
| remote_ip_prefix  | 192.168.1.0/24                       |
| revision_number   | 0                                    |
| security_group_id | 142c190e-2211-447f-9a20-fe86d5d40b60 |
| updated_at        | 2018-08-14T02:59:38Z                 |
+-------------------+--------------------------------------+

为指定网段创建 TCP 端口出口规则

[root@10e150e68e69 ~]# openstack security group rule create --protocol tcp --remote-ip 192.168.1.0/24 --egress --dst-port 80 ops_test
+-------------------+--------------------------------------+
| Field             | Value                                |
+-------------------+--------------------------------------+
| created_at        | 2018-08-14T03:15:04Z                 |
| description       |                                      |
| direction         | egress                               |
| ether_type        | IPv4                                 |
| id                | 4a298819-9b65-43c7-977f-baf74de07053 |
| name              | None                                 |
| port_range_max    | 80                                   |
| port_range_min    | 80                                   |
| project_id        | 72fc263931ec46529f18ed7450432fe7     |
| protocol          | tcp                                  |
| remote_group_id   | None                                 |
| remote_ip_prefix  | 192.168.1.0/24                       |
| revision_number   | 0                                    |
| security_group_id | 142c190e-2211-447f-9a20-fe86d5d40b60 |
| updated_at        | 2018-08-14T03:15:04Z                 |
+-------------------+--------------------------------------+

安全组编辑

修改安全组描述和名称

[root@10e150e68e69 ~]# openstack security group set --name ops_test1 --description ops_test1 ops_test

验证

[root@10e150e68e69 ~]# openstack security group show ops_test1
+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field           | Value                                                                                                                                                                                                                                               |
+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| created_at      | 2018-08-13T07:38:42Z                                                                                                                                                                                                                                |
| description     | ops_test1                                                                                                                                                                                                                                           |
| id              | 142c190e-2211-447f-9a20-fe86d5d40b60                                                                                                                                                                                                                |
| name            | ops_test1                                                                                                                                                                                                                                           |
| project_id      | 72fc263931ec46529f18ed7450432fe7                                                                                                                                                                                                                    |
| revision_number | 4                                                                                                                                                                                                                                                   |
| rules           | created_at='2018-08-13T07:38:42Z', direction='ingress', ethertype='IPv4', id='22f55388-d654-4ef4-a04d-874df7757244', remote_group_id='142c190e-2211-447f-9a20-fe86d5d40b60', updated_at='2018-08-13T07:38:42Z'                                      |
|                 | created_at='2018-08-13T07:43:46Z', direction='ingress', ethertype='IPv4', id='96c58083-20db-4e2b-abc7-cd90c905f170', port_range_max='80', port_range_min='80', protocol='tcp', remote_ip_prefix='192.168.1.0/24', updated_at='2018-08-13T07:43:46Z' |
|                 | created_at='2018-08-13T07:38:42Z', direction='egress', ethertype='IPv4', id='c60f7349-4306-44ad-bced-5890131a0341', updated_at='2018-08-13T07:38:42Z'                                                                                               |
| updated_at      | 2018-08-13T08:02:42Z                                                                                                                                                                                                                                |
+-----------------+----------------------------------------------------------------------------------------

安全组删除

[root@10e150e68e69 ~]# openstack security group delete 142c190e-2211-447f-9a20-fe86d5d40b60

验证

[root@10e150e68e69 ~]# openstack security group list |grep 142c190e-2211-447f-9a20-fe86d5d40b60

安全组规则删除

[root@10e150e68e69 ~]# openstack security group rule delete 96c58083-20db-4e2b-abc7-cd90c905f170

验证

[root@10e150e68e69 ~]# openstack security group rule list ops_test1
+--------------------------------------+-------------+----------+------------+--------------------------------------+
| ID                                   | IP Protocol | IP Range | Port Range | Remote Security Group                |
+--------------------------------------+-------------+----------+------------+--------------------------------------+
| 22f55388-d654-4ef4-a04d-874df7757244 | None        | None     |            | 142c190e-2211-447f-9a20-fe86d5d40b60 |
| c60f7349-4306-44ad-bced-5890131a0341 | None        | None     |            | None                                 |
+--------------------------------------+-------------+----------+------------+------------------------------

查询dhcp所在节点

usage: neutron dhcp-agent-list-hosting-net [-h]
                                        [-f {csv,json,table,value,yaml}]
                                        [-c COLUMN] [--max-width <integer>]
                                        [--fit-width] [--print-empty]
                                        [--noindent]
                                        [--quote {all,minimal,none,nonnumeric}]
                                        [--sort-column SORT_COLUMN] [-D]
                                        [-F FIELD]
                                        NETWORK

验证

[root@hb02-other-172e28e8e132 ~]# neutron dhcp-agent-list-hosting-net d06209ce-5a47-4ccf-8061-4a670f5f4b1f
+--------------------------------------+-------------------------+----------------+-------+
| id                                   | host                    | admin_state_up | alive |
+--------------------------------------+-------------------------+----------------+-------+
| 41255f46-3795-483a-bc6b-fc02b0efaa1e | hb02-other-172e28e8e139 | True           | :-)   |
| bd96e543-b28a-4ff3-9889-fe973af12e7d | hb02-other-172e28e8e135 | True           | :-)   |
+--------------------------------------+-------------------------+----------------+-------+

添加dhcp-agent

usage: neutron dhcp-agent-network-add [-h] DHCP_AGENT NETWORK

Add a network to a DHCP agent.
positional arguments:
  DHCP_AGENT  ID of the DHCP agent.
  NETWORK     Network to add.
optional arguments:
  -h, --help  show this help message and exit

验证

neutron dhcp-agent-network-add 08df9830-4b2d-4c8e-8539-6d14e6dcdea9 d06209ce-5a47-4ccf-8061-4a670f5f4b1f
+--------------------------------------+-------------------------+----------------+-------+
| id                                   | host                    | admin_state_up | alive |
+--------------------------------------+-------------------------+----------------+-------+
| 08df9830-4b2d-4c8e-8539-6d14e6dcdea9 | hb02-other-172e28e8e136 | True           | :-)   |
| 41255f46-3795-483a-bc6b-fc02b0efaa1e | hb02-other-172e28e8e139 | True           | :-)   |
| bd96e543-b28a-4ff3-9889-fe973af12e7d | hb02-other-172e28e8e135 | True           | :-)   |
+--------------------------------------+-------------------------+----------------+-------+

删除dhcp-agent

usage: neutron dhcp-agent-network-remove [-h] DHCP_AGENT NETWORK

Remove a network from a DHCP agent.
positional arguments:
  DHCP_AGENT  ID of the DHCP agent.
  NETWORK     Network to remove.
optional arguments:
  -h, --help  show this help message and exit

验证

 [root@hb02-other-172e28e8e132 ~]# neutron dhcp-agent-network-remove  08df9830-4b2d-4c8e-8539-6d14e6dcdea9 d06209ce-5a47-4ccf-8061-4a670f5f4b1f

+--------------------------------------+-------------------------+----------------+-------+
| id                                   | host                    | admin_state_up | alive |
+--------------------------------------+-------------------------+----------------+-------+
| 41255f46-3795-483a-bc6b-fc02b0efaa1e | hb02-other-172e28e8e139 | True           | :-)   |
| bd96e543-b28a-4ff3-9889-fe973af12e7d | hb02-other-172e28e8e135 | True           | :-)   |
+--------------------------------------+-------------------------+----------------+-------+

故障处理

问题1:虚拟机内部无法获取 IP

问题现象

通过 OpenStack 命令创建的虚拟机,登录虚拟机后,虚拟机内部没有获取到 IP

处理过程

  1. 查看虚拟机 ID:4983dbf2-5dd3-4790-b9e8-938170d50a01
  2. 通过 VNC 登录虚拟机
# openstack console url show 4983dbf2-5dd3-4790-b9e8-938170d50a01

1-1-url-show

手动获取不到 IP

1-2-dhclient

  1. 获取虚拟机的 Port ID 以及所在的网络 ID
# nova interface-list 4983dbf2-5dd3-4790-b9e8-938170d50a01

1-3-interface-list

网络 ID:59791ba5-dba7-4454-9575-29b3096bf8cf

虚拟机的 tap 设备:tap80809a14-7d(Port ID 的前 11 位)

  1. 获取 dhcp agent 所在的主机
# openstack network agent list --network 59791ba5-dba7-4454-9575-29b3096bf8cf

1-4-agent-list

dhcp agent 所在的主机 10e131e73e27 和 10e131e73e22

  1. 查找虚拟机所在的计算节点
# nova show 4983dbf2-5dd3-4790-b9e8-938170d50a01

计算节点 10e131e73e24

在计算节点查看 vxlan 设备,找到对应 dhcp agent 所在主机的 vxlan 设备

# ovs-vsctl show|grep vxlan -C 1

1-5-vxlan1

1-6-vxlan2

记录设备名称:vxlan-0a834816 和 vxlan-0a83481b

  1. 查看 vxlan 设备对应的 Port
# ovs-ofctl show br-tun|grep vxlan-0a834816
24(vxlan-0a834816): addr:3e:21:a8:fc:59:32
ofport:24
# ovs-ofctl show br-tun|grep vxlan-0a83481b
27(vxlan-0a83481b): addr:32:17:2a:b9:75:50
ofport:27
  1. 查看虚拟机 tap 设备对应的 tag
# ovs-vsctl show|grep tap80809a14-7d -C 1

1-7-tag

  1. 查看虚拟机所属 vlan 的流表
# ovs-ofctl dump-flows br-tun|grep dl_vlan=5

查看 table22 (广播)的流表

# ovs-ofctl dump-flows br-tun|grep dl_vlan=5|grep table=22

没有广播到 vxlan 端口 24 和 27,导致虚拟机无法获取 IP

  1. 重启 ovs-agent,重新获取流表后可以手动获取到 IP
# systemctl restart neutron-openvswitch-agent

问题原因

相关流表丢失,导致虚拟机获取 IP 无法广播到对应的 dhcp agent,最终无法获取 IP。

问题2:OpenStack 同一子网下的虚拟机通过 fixed-ip SSH 登录失败

问题现象

同一子网下的虚拟机之间通过浮动 IP 可以 SSH 登录,但是不能通过 fixed-ip SSH 登录。

处理过程

  1. 通过查看,这两台虚拟机IP地址:10.10.10.6 和 10.10.10.9,部署在不同的计算节点,管理网通过 VXLAN 连接,互通测试情况:通过浮动 IP 可以相互 ping 通,也可以相互 SSH 连接;通过内网 IP 可以相互 ping 通,不可以相互 SSH 连接。
  2. 查看 sshd 服务运行正常,22 端口开启,排除是虚拟机的问题。
  3. 查看虚拟机的 MTU 是 1500,而两个计算节点的出口网卡 MTU 也是 1500,推测是两个虚拟机通过 VXLAN 通信,但没有为 VXLAN 预留需要的长度出现的问题。

2-1-mtu

  1. 将计算节点的 MTU 值改为 1550,命令:ifconfig 网卡名称 mtu 1550。
  2. 虚拟机之间通过内网 IP,可以正常 SSH 登录。

问题原因

两个虚拟机部署在不同的计算节点,通过内网通信需要通过 VXLAN 建立隧道,而 VXLAN 通信打标签需要留出额外50字节长度(14 字节 MAC 头(虚拟机网卡添加) + 8 字节 VXLAN 头 + 8 字节 UDP 头 + 20 字节 IP 头)

2-2-vxlan

两台虚拟机建立 SSH 连接时候,发出有大于 1450 的包,这个包出了虚拟机后加上 50 的 VXLAN 包头,就会因为超出 1500 的 MTU 值而出不去。

PS:由于 ping 命令是使用 64 位的小包,所以使用内网地址互相 ping,不会受到影响。另外,通过浮动 IP 连接没有走 VXLAN 的网络,所以可以 ping 通,也可以 SSH 正常登录。

问题3:虚拟机多网卡环境,通过内网 IP ping 互通,floating-ip ping 不通

问题现象

虚拟机多网卡环境,通过内网 IP ping 互通,floating-ip ping 不通

处理过程

  1. 进虚拟机所在计算节点的 fip ns 里面 ping floating-ip 不通,说明是 OpenStack 环境内部问题。
  2. 分别查看 fip ns 里面的 route 规则和 qroute ns 里面的 iptables nat 规则,都正常。
  3. 从 qroute ns 里面 ping 虚拟机 fix ip,是可以通的。
  4. 从 console 登录虚拟机,查看默认路由发现有两条。
  5. 删除其中一个网络的默认网关,重启虚拟机后问题解决。

问题原因

虚拟机同时连了两个网络,两个网络都给分了默认路由,导致默认路由混乱所致。

问题4:虚机突然失联,无法 ping 通和 SSH 登录

问题现象

马驹桥 Bss10 省测试集群,收到 zabbix 监控告警,虚机失联超过 5 分钟。

处理过程

  1. 检查虚机状态,显示为 active。
  2. ping 或 SSH 均无法访问虚机,考虑虚机网络出现问题。
  3. 通过 console 访问虚机,检查虚机的 IP,发现该虚机只有 lo 地址。
  4. 马驹桥镜像是通过 NetworkManager 进程来管理 IP 的,查看 NetworkManager 的进程状态,发现 NetworkManager 进程已经 down 掉。
  5. 重启 NetworkManager 进程。
  6. 通过 ip a 命令,发现虚拟重新获取 IP,并可以正常访问。

问题原因

虚机 IP 管理依赖 NetworkManager 进程,NetworkManager 进程关闭,导致网卡 eth0 处于 down 的状态。

问题5:升级 neutron 组件后,发现浮动 IP 无法连接

问题现象

升级 neutron 组件后,用户反映浮动 IP 无法连接。

处理过程

  1. 通过 nova list 查看虚拟机,发现浮动 IP 正常绑定。
  2. ping 浮动IP,无法 ping 通。
  3. 从同一网段的其它虚拟机,ping 该虚拟机的内网地址,可以 ping 通,判断虚拟机正常,浮动 IP 相关服务异常。
  4. 通过 neutron agent-list 查看,发现 agent 正常。
  5. 进入到虚拟机所在的节点,通过 systemctl status neutron* 查看进程的状态,发现各服务正常。
  6. 查看该物理节点的 /var/log/neutron/l3-agent.log 日志,发现如下错误

5-2-l3-agent-error

(1146, u”Table ‘neutron.qos_router_gw_policy_bindings’ doesn’t exist”),数据库表不存在。

  1. 执行升级数据库。
# neutron-db-manage --config-file /etc/neutron/neutron.conf upgrade 195176fb410d
  1. 重启 neutron-l3-agent 进程,问题解决。

问题原因

neutron 组件新版本增加了新的数据库表,升级时没有升级数据库,导致 l3-agent 服务异常。

问题6:在验收郴州资源池时,出现公网不通。

问题现象

创建的虚拟机内网地址是 192.168.30.9,绑定的公网 ip 是 175.6.251.35,snat 空间的内网地址是 192.168.30.8,公网地址是绑定的公网 ip。 发现配置的公网 IP,在两个网络节点的 snat namespace 里的 sg 口上都配上了 IP 地址 175.6.251.35,且虚拟机 ping 不通网络节点上 snat namespace 中的 sg 口 192.168.30.8。

处理过程

  1. 查看两个网络节点上 l3 agent的ha状态都是 active,
# neutron l3-agent-list-hosting-router 9381dde9-7a8a-48d0-8e6d-b2046a8f475a
+--------------------------------------+--------------------------+----------------+-------+----------+
| id                                   | host                     | admin_state_up | alive | ha_state |
+--------------------------------------+--------------------------+----------------+-------+----------+
| 9d2f1d10-ee3c-4344-8704-39becd010685 | hn02-network-10e116e1e16 | True           | :-)   | active   |
| b0dc7298-092a-458f-bc74-aaffee2a34a2 | hn02-network-10e116e1e18 | True           | :-)   | active   |
+--------------------------------------+--------------------------+----------------+-------+----------+
  1. 两个 snat namespace 的 HA 口互 ping,ping 不通,怀疑两个网络节点不通。
  2. 发现两个网络节点是可以通的,检查 HA 网络的类型。
  3. HA 网络的类型是 VXLAN,那就检查隧道是否通。
  4. 通过 ovs-vsctl show,VXLAN 的隧道 IP 地址是 10.165.7.0 网段,两个网络节点这个网段也是可以 ping 通的,怀疑是两个节点的包被 iptables 丢掉了。
  5. 在两个网络节点上 iptables -S,发现 VXLAN 的 udp 端口 4789 没有放开。
  6. 放开端口,恢复正常。
  7. 计算节点上 iptables -S,发现 VXLAN 的 udp 端口 4789 没有放开。
  8. 在计算节点上添加 iptables 规则,允许 udp 端口是 4789 的包通过,恢复正常。

问题原因

计算节点和网络节点都没配置 vxlan 的端口,导致 2 个网络节点 ha 网络不能通信,因此 2 个网络节点都认为自己是 master,计算节点和网络节点之间 vxlan 端口不通, 导致虚拟机 ping 不通网络节点的 snat。

问题7:马驹桥资源池 vnc 连接不上

问题现象

所有虚拟机的 vnc 都连接不上。

处理过程

  1. 查看 2 个 vnc 节点的 haproxy,keepalived,novncproxy 状态、日志以及配置文件,均正常。
  2. 查看相关的端口都正常启动。
  3. 排查网络方面的问题,检查网卡的状态,ip 配置,路由信息。
  4. 发现网关设置错误,修改为正确的值,问题解决。
# cat /etc/sysconfig/network-scripts/ifcfg-eth0
GATEWAY=x.x.x.x

问题原因

网关设置错误。