3.5 Pod的配置管理
应用部署的一个最佳实践是将应用所需的配置信息与程序进行分离,这样可以使应用程序被更好地复用,通过不同的配置也能实现更灵活的功能。将应用打包为容器镜像后,可以通过环境变量或者外挂文件的方式在创建容器时进行配置注入,但在大规模容器集群的环境中,对多个容器进行不同的配置将变得非常复杂。从Kubernetes 1.2开始提供了一种统一的应用配置管理方案——ConfigMap。本节对ConfigMap的概念和用法进行详细描述。
3.5.1 ConfigMap概述
ConfigMap供容器使用的典型用法如下。
(1)生成为容器内的环境变量。
(2)设置容器启动命令的启动参数(需设置为环境变量)。
(3)以Volume的形式挂载为容器内部的文件或目录。
ConfigMap以一个或多个key:value的形式保存在Kubernetes系统中供应用使用,既可以用于表示一个变量的值(例如apploglevel=info),也可以用于表示一个完整配置文件的内容(例如server.xml=<?xml...>...)
可以通过YAML配置文件或者直接使用kubectl create configmap命令行的方式来创建ConfigMap。
3.5.2 创建ConfigMap资源对象
1.通过YAML配置文件方式创建
下面的例子cm-appvars.yaml描述了将几个应用所需的变量定义为ConfigMap的用法:
cm-appvars.yaml apiVersion: v1 kind: ConfigMap metadata: name: cm-appvars data: apploglevel: info appdatadir: /var/data
执行kubectl create命令创建该ConfigMap:
$kubectl create -f cm-appvars.yaml configmap "cm-appvars" created
查看创建好的ConfigMap:
# kubectl get configmap NAME DATA AGE cm-appvars 2 3s # kubectl describe configmap cm-appvars Name: cm-appvars Namespace: default Labels: <none> Annotations: <none> Data ==== appdatadir: 9 bytes apploglevel: 4 bytes # kubectl get configmap cm-appvars -o yaml apiVersion: v1 data: appdatadir: /var/data apploglevel: info kind: ConfigMap metadata: creationTimestamp: 2016-07-28T19:57:16Z name: cm-appvars namespace: default resourceVersion: "78709" selfLink: /api/v1/namespaces/default/configmaps/cm-appvars uid: 7bb2e9c0-54fd-11e6-9dcd-000c29dc2102
下面的例子cm-appconfigfiles.yaml描述了将两个配置文件server.xml和logging.properties定义为ConfigMap的用法,设置key为配置文件的别名,value则是配置文件的全部文本内容:
cm-appconfigfiles.yaml apiVersion: v1 kind: ConfigMap metadata: name: cm-appconfigfiles data: key-serverxml: | <?xml version='1.0' encoding='utf-8'?> <Server port="8005" shutdown="SHUTDOWN"> <Listener className="org.apache.catalina.startup.VersionLoggerListener" /> <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" /> <Listener className= "org.apache.catalina.core.JreMemoryLeakPreventionListener" /> <Listener className= "org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" /> <Listener className= "org.apache.catalina.core.ThreadLocalLeakPreventionListener" /> <GlobalNamingResources> <Resource name="UserDatabase" auth="Container" type="org.apache.catalina.UserDatabase" description="User database that can be updated and saved" factory="org.apache.catalina.users.MemoryUserDatabaseFactory" pathname="conf/tomcat-users.xml" /> </GlobalNamingResources> <Service name="Catalina"> <Connector port="8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443" /> <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" /> <Engine name="Catalina" defaultHost="localhost"> <Realm className="org.apache.catalina.realm.LockOutRealm"> <Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase"/> </Realm> <Host name="localhost" appBase="webapps" unpackWARs="true" autoDeploy="true"> <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" prefix="localhost_access_log" suffix=".txt" pattern="%h %l %u %t "%r" %s %b" /> </Host> </Engine> </Service> </Server> key-loggingproperties:"handlers =1catalina.org.apache.juli.FileHandler, 2localhost.org.apache.juli. FileHandler, 3manager.org.apache.juli.FileHandler, 4host-manager.org.apache.juli. FileHandler, java.util.logging.ConsoleHandler\r\n\r\n.handlers= 1catalina.org.apache. juli.FileHandler, java.util.logging.ConsoleHandler\r\n\r\n1catalina.org.apache.juli.FileHandler.level = FINE\r\n1catalina.org.apache.juli.FileHandler.directory = ${catalina.base}/logs\r\n1catalina.org.apache.juli.FileHandler.prefix = catalina.\r\n\r\n2localhost.org.apache.juli.FileHandler.level = FINE\r\n2localhost.org.apache.juli.FileHandler.directory = ${catalina.base}/logs\r\n2localhost.org.apache.juli.FileHandler.prefix = localhost.\r\n\r\n3manager.org.apache.juli.FileHandler.level = FINE\r\n3manager.org.apache.juli.FileHandler.directory = ${catalina.base}/logs\r\n3manager.org.apache.juli.FileHandler.prefix = manager.\r\n\r\n4host-manager.org.apache.juli.FileHandler.level = FINE\r\n4host-manager.org.apache.juli.FileHandler.directory = ${catalina.base}/logs\r\n4host-manager.org.apache.juli.FileHandler. prefix = host-manager.\r\n\r\njava.util.logging.ConsoleHandler.level = FINE\r\ njava.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter\r\n\r\n\r\norg.apache.catalina.core. ContainerBase.[Catalina].[localhost].level = INFO\r\norg.apache.catalina.core.ContainerBase.[Catalina].[localhost]. handlers = 2localhost.org.apache.juli.FileHandler\r\n\r\norg.apache.catalina.core. ContainerBase.[Catalina].[localhost].[/manager].level = INFO\r\norg.apache.catalina.core.ContainerBase.[Catalina].[localhost]. [/manager].handlers = 3manager.org.apache.juli.FileHandler\r\n\r\norg.apache.catalina.core. ContainerBase.[Catalina].[localhost].[/host-manager].level = INFO\r\norg.apache.catalina.core.ContainerBase.[Catalina].[localhost]. [/host-manager].handlers = 4host-manager.org.apache.juli.FileHandler\r\n\r\n"
执行kubectl create命令创建该ConfigMap:
$kubectl create -f cm-appconfigfiles.yaml configmap "cm-appconfigfiles" created
查看创建好的ConfigMap:
# kubectl get configmap cm-appconfigfiles NAME DATA AGE cm-appconfigfiles 2 14s # kubectl describe configmap cm-appconfigfiles Name: cm-appconfigfiles Namespace: default Labels: <none> Annotations: <none> Data ==== key-loggingproperties: 1809 bytes key-serverxml: 1686 bytes
查看已创建的ConfigMap的详细内容,可以看到两个配置文件的全文:
# kubectl get configmap cm-appconfigfiles -o yaml apiVersion: v1 data: key-loggingproperties:"handlers = 1catalina.org.apache.juli.FileHandler, 2localhost.org.apache.juli.FileHandler, 3manager.org.apache.juli.FileHandler, 4host-manager.org.apache.juli. FileHandler, java.util.logging.ConsoleHandler\r\n\r\n.handlers = 1catalina.org.apache. juli.FileHandler, java.util.logging.ConsoleHandler\r\n\r\n1catalina.org.apache.juli. FileHandler.level = FINE\r\n1catalina.org.apache.juli.FileHandler.directory = ${catalina.base}/logs\r\n1catalina.org.apache.juli.FileHandler.prefix = catalina.\r\n\r\n2localhost.org.apache.juli.FileHandler.level = FINE\r\n2localhost.org.apache.juli.FileHandler.directory = ${catalina.base}/logs\r\n2localhost.org.apache.juli.FileHandler.prefix = localhost.\r\n\r\n3manager.org.apache.juli.FileHandler.level = FINE\r\n3manager.org.apache.juli.FileHandler.directory = ${catalina.base}/logs\r\n3manager.org.apache.juli.FileHandler.prefix = manager.\r\n\r\n4host-manager.org.apache.juli.FileHandler.level = FINE\r\n4host-manager.org.apache.juli.FileHandler.directory = ${catalina.base}/logs\r\n4host-manager.org.apache.juli.FileHandler. prefix = host-manager.\r\n\r\njava.util.logging.ConsoleHandler.level = FINE\r\njava. util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter\r\n\r\n\r\norg.apache.catalina.core. ContainerBase.[Catalina].[localhost].level = INFO\r\norg.apache.catalina.core.ContainerBase.[Catalina].[localhost]. handlers = 2localhost.org.apache.juli.FileHandler\r\n\r\norg.apache.catalina.core. ContainerBase.[Catalina].[localhost].[/manager].level = INFO\r\norg.apache.catalina.core.ContainerBase.[Catalina].[localhost]. [/manager].handlers = 3manager.org.apache.juli.FileHandler\r\n\r\norg.apache.catalina.core. ContainerBase.[Catalina].[localhost].[/host-manager].level = INFO\r\norg.apache.catalina.core.ContainerBase.[Catalina].[localhost]. [/host-manager].handlers = 4host-manager.org.apache.juli.FileHandler\r\n\r\n" key-serverxml: | <?xml version='1.0' encoding='utf-8'?> <Server port="8005" shutdown="SHUTDOWN"> <Listener className="org.apache.catalina.startup.VersionLoggerListener" /> <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" /> <Listener className="org.apache.catalina.core. JreMemoryLeakPreventionListener" /> <Listener className="org.apache.catalina.mbeans. GlobalResourcesLifecycleListener" /> <Listener className="org.apache.catalina.core. ThreadLocalLeakPreventionListener" /> <GlobalNamingResources> <Resource name="UserDatabase" auth="Container" type="org.apache.catalina.UserDatabase" description="User database that can be updated and saved" factory="org.apache.catalina.users.MemoryUserDatabaseFactory" pathname="conf/tomcat-users.xml" /> </GlobalNamingResources> <Service name="Catalina"> <Connector port="8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443" /> <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" /> <Engine name="Catalina" defaultHost="localhost"> <Realm className="org.apache.catalina.realm.LockOutRealm"> <Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase"/> </Realm> <Host name="localhost" appBase="webapps" unpackWARs="true" autoDeploy="true"> <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" prefix="localhost_access_log" suffix=".txt" pattern="%h %l %u %t "%r" %s %b" /> </Host> </Engine> </Service> </Server> kind: ConfigMap metadata: creationTimestamp: 2016-07-29T00:52:18Z name: cm-appconfigfiles namespace: default resourceVersion: "85054" selfLink: /api/v1/namespaces/default/configmaps/cm-appconfigfiles uid: b30d5019-5526-11e6-9dcd-000c29dc2102
2.通过kubectl命令行方式创建
不使用YAML文件,直接通过kubectl create configmap也可以创建ConfigMap,可以使用参数--from-file或--from-literal指定内容,并且可以在一行命令中指定多个参数。
(1)通过--from-file参数从文件中进行创建,可以指定key的名称,也可以在一个命令行中创建包含多个key的ConfigMap,语法为:
# kubectl create configmap NAME --from-file=[key=]source --from-file=[key=]source
(2)通过--from-file参数从目录中进行创建,该目录下的每个配置文件名都被设置为key,文件的内容被设置为value,语法为:
# kubectl create configmap NAME --from-file=config-files-dir
(3)使用--from-literal时会从文本中进行创建,直接将指定的key#=value#创建为ConfigMap的内容,语法为:
# kubectl create configmap NAME --from-literal=key1=value1--from-literal= key2=value2
下面对这几种用法举例说明。
例如,在当前目录下含有配置文件server.xml,可以创建一个包含该文件内容的ConfigMap:
# kubectl create configmap cm-server.xml --from-file=server.xml
configmap "cm-server.xml" created
# kubectl describe configmap cm-server.xml
Name: cm-server.xml
Namespace: default
Labels: <none>
Annotations: <none>
Data
====
server.xml: 6458 bytes
假设在configfiles目录下包含两个配置文件server.xml和logging.properties,创建一个包含这两个文件内容的ConfigMap:
# kubectl create configmap cm-appconf --from-file=configfiles
configmap "cm-appconf" created
# kubectl describe configmap cm-appconf
Name: cm-appconf
Namespace: default
Labels: <none>
Annotations: <none>
Data
====
logging.properties: 3354 bytes
server.xml: 6458 bytes
使用--from-literal参数进行创建的示例如下:
# kubectl create configmap cm-appenv --from-literal=loglevel=info --from-literal =appdatadir=/var/data configmap "cm-appenv" created # kubectl describe configmap cm-appenv Name: cm-appenv Namespace: default Labels: <none> Annotations: <none> Data ==== appdatadir: 9 bytes loglevel: 4 bytes
容器应用对ConfigMap的使用有以下两种方法。
(1)通过环境变量获取ConfigMap中的内容。
(2)通过Volume挂载的方式将ConfigMap中的内容挂载为容器内部的文件或目录。
3.5.3 在Pod中使用ConfigMap
1.通过环境变量方式使用ConfigMap
以前面创建的ConfigMap“cm-appvars”为例:
apiVersion: v1 kind: ConfigMap metadata: name: cm-appvars data: apploglevel: info appdatadir: /var/data
在Pod“cm-test-pod”的定义中,将ConfigMap“cm-appvars”中的内容以环境变量(APPLOGLEVEL和APPDATADIR)方式设置为容器内部的环境变量,容器的启动命令将显示这两个环境变量的值("env | grep APP"):
apiVersion: v1 kind: Pod metadata: name: cm-test-pod spec: containers: - name: cm-test image: busybox command: [ "/bin/sh", "-c", "env | grep APP" ] env: - name: APPLOGLEVEL # 定义环境变量的名称 valueFrom: # key“apploglevel”对应的值 configMapKeyRef: name: cm-appvars # 环境变量的值取自cm-appvars: key: apploglevel # key为apploglevel - name: APPDATADIR # 定义环境变量的名称 valueFrom: # key“appdatadir”对应的值 configMapKeyRef: name: cm-appvars # 环境变量的值取自cm-appvars key: appdatadir # key为appdatadir restartPolicy: Never
使用kubectl create -f命令创建该Pod,由于是测试Pod,所以该Pod在执行完启动命令后将会退出,并且不会被系统自动重启(restartPolicy=Never):
# kubectl create -f cm-test-pod.yaml pod "cm-test-pod" created
使用kubectl get pods --show-all查看已经停止的Pod:
# kubectl get pods --show-all NAME READY STATUS RESTARTS AGE cm-test-pod 0/1 Completed 0 8s
查看该Pod的日志,可以看到启动命令“env | grep APP”的执行结果如下:
# kubectl logs cm-test-pod APPDATADIR=/var/data APPLOGLEVEL=info
说明容器内部的环境变量使用ConfigMap cm-appvars中的值进行了正确设置。
Kubernetes从1.6版本开始,引入了一个新的字段envFrom,实现了在Pod环境中将ConfigMap(也可用于Secret资源对象)中所有定义的key=value自动生成为环境变量:
apiVersion: v1 kind: Pod metadata: name: cm-test-pod spec: containers: - name: cm-test image: busybox command: [ "/bin/sh", "-c", "env" ] envFrom: - configMapRef name: cm-appvars # 根据cm-appvars中的key=value自动生成环境变量 restartPolicy: Never
通过这个定义,在容器内部将会生成如下环境变量:
apploglevel=info appdatadir=/var/data
需要说明的是,环境变量的名称受POSIX命名规范([a-zA-Z_][a-zA-Z0-9_]*)约束,不能以数字开头。如果包含非法字符,则系统将跳过该条环境变量的创建,并记录一个Event来提示环境变量无法生成,但并不阻止Pod的启动。
2.通过volumeMount使用ConfigMap
在如下所示的cm-appconfigfiles.yaml例子中包含两个配置文件的定义:server.xml和logging.properties。
cm-appconfigfiles.yaml apiVersion: v1 kind: ConfigMap metadata: name: cm-serverxml data: key-serverxml: | <?xml version='1.0' encoding='utf-8'?> <Server port="8005" shutdown="SHUTDOWN"> <Listener className="org.apache.catalina.startup.VersionLoggerListener" /> <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" /> <Listener className="org.apache.catalina.core. JreMemoryLeakPreventionListener" /> <Listener className="org.apache.catalina.mbeans. GlobalResourcesLifecycleListener" /> <Listener className="org.apache.catalina.core. ThreadLocalLeakPreventionListener" /> <GlobalNamingResources> <Resource name="UserDatabase" auth="Container" type="org.apache.catalina.UserDatabase" description="User database that can be updated and saved" factory="org.apache.catalina.users.MemoryUserDatabaseFactory" pathname="conf/tomcat-users.xml" /> </GlobalNamingResources> <Service name="Catalina"> <Connector port="8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443" /> <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" /> <Engine name="Catalina" defaultHost="localhost"> <Realm className="org.apache.catalina.realm.LockOutRealm"> <Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase"/> </Realm> <Host name="localhost" appBase="webapps" unpackWARs="true" autoDeploy="true"> <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" prefix="localhost_access_log" suffix=".txt" pattern="%h %l %u %t "%r" %s %b" /> </Host> </Engine> </Service> </Server> key-loggingproperties:"handlers = 1catalina.org.apache.juli.FileHandler, 2localhost.org.apache.juli.FileHandler, 3manager.org.apache.juli.FileHandler, 4host-manager.org.apache.juli.FileHandler, java.util.logging.ConsoleHandler\r\n\r\n.handlers = 1catalina.org.apache.juli.FileHandler, java.util.logging.ConsoleHandler\r\n\r\n1catalina.org.apache.juli.FileHandler.level = FINE\r\n1catalina.org.apache.juli.FileHandler.directory = ${catalina.base}/logs\r\n1catalina.org.apache.juli.FileHandler.prefix = catalina.\r\n\r\n2localhost.org.apache.juli.FileHandler.level = FINE\r\n2localhost.org.apache.juli.FileHandler.directory = ${catalina.base}/logs\r\n2localhost.org.apache.juli.FileHandler.prefix = localhost.\r\n\r\n3manager.org.apache.juli.FileHandler.level = FINE\r\n3manager.org.apache.juli.FileHandler.directory = ${catalina.base}/logs\r\n3manager.org.apache.juli.FileHandler.prefix = manager.\r\n\r\n4host-manager.org.apache.juli.FileHandler.level = FINE\r\n4host-manager.org.apache.juli.FileHandler.directory = ${catalina.base}/logs\r\n4host-manager.org.apache.juli.FileHandler. prefix = host-manager.\r\n\r\njava.util.logging.ConsoleHandler.level = FINE\r\njava.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter\r\n\r\n\r\norg.apache.catalina.core. ContainerBase.[Catalina].[localhost].level = INFO\r\norg.apache.catalina.core.ContainerBase.[Catalina].[localhost]. handlers = 2localhost.org.apache.juli.FileHandler\r\n\r\norg.apache.catalina.core. ContainerBase.[Catalina].[localhost].[/manager].level = INFO\r\norg.apache.catalina.core.ContainerBase.[Catalina].[localhost]. [/manager].handlers = 3manager.org.apache.juli.FileHandler\r\n\r\norg.apache.catalina.core. ContainerBase.[Catalina].[localhost].[/host-manager].level = INFO\r\norg.apache.catalina.core.ContainerBase.[Catalina].[localhost]. [/host-manager].handlers = 4host-manager.org.apache.juli.FileHandler\r\n\r\n"
在Pod“cm-test-app”的定义中,将ConfigMap“cm-appconfigfiles”中的内容以文件的形式mount到容器内部的/configfiles目录下。Pod配置文件cm-test-app.yaml的内容如下:
apiVersion: v1 kind: Pod metadata: name: cm-test-app spec: containers: - name: cm-test-app image: kubeguide/tomcat-app:v1 ports: - containerPort: 8080 volumeMounts: - name: serverxml # 引用Volume的名称 mountPath: /configfiles # 挂载到容器内的目录 volumes: - name: serverxml # 定义Volume的名称 configMap: name: cm-appconfigfiles # 使用ConfigMap“cm-appconfigfiles” items: - key: key-serverxml # key=key-serverxml path: server.xml # value将server.xml文件名进行挂载 - key: key-loggingproperties # key=key-loggingproperties path: logging.properties # value将logging.properties文件名进行挂载
创建该Pod:
# kubectl create -f cm-test-app.yaml pod "cm-test-app" created
登录容器,查看到在/configfiles目录下存在server.xml和logging.properties文件,它们的内容就是ConfigMap“cm-appconfigfiles”中两个key定义的内容:
# kubectl exec -ti cm-test-app -- bash root@cm-test-app:/# cat /configfiles/server.xml <?xml version='1.0' encoding='utf-8'?> <Server port="8005" shutdown="SHUTDOWN"> ...... root@cm-test-app:/# cat /configfiles/logging.properties handlers = 1catalina.org.apache.juli.AsyncFileHandler, 2localhost.org.apache.juli.AsyncFileHandler, 3manager.org.apache.juli.AsyncFileHandler, 4host-manager.org.apache.juli.AsyncFileHandler, java.util.logging.ConsoleHandler ......
如果在引用ConfigMap时不指定items,则使用volumeMount方式在容器内的目录下为每个item都生成一个文件名为key的文件。
Pod配置文件cm-test-app.yaml的内容如下:
apiVersion: v1 kind: Pod metadata: name: cm-test-app spec: containers: - name: cm-test-app image: kubeguide/tomcat-app:v1 imagePullPolicy: Never ports: - containerPort: 8080 volumeMounts: - name: serverxml # 引用Volume的名称 mountPath: /configfiles # 挂载到容器内的目录 volumes: - name: serverxml # 定义Volume的名称 configMap: name: cm-appconfigfiles # 使用ConfigMap“cm-appconfigfiles”
创建该Pod:
# kubectl create -f cm-test-app.yaml pod "cm-test-app" created
登录容器,查看到在/configfiles目录下存在key-loggingproperties和key-serverxml文件,文件的名称来自在ConfigMap cm-appconfigfiles中定义的两个key的名称,文件的内容则为value的内容:
# ls /configfiles key-loggingproperties key-serverxml
3.5.4 使用ConfigMap的限制条件
使用ConfigMap的限制条件如下。
◎ ConfigMap必须在Pod之前创建。
◎ ConfigMap受Namespace限制,只有处于相同Namespace中的Pod才可以引用它。
◎ ConfigMap中的配额管理还未能实现。
◎ kubelet只支持可以被API Server管理的Pod使用ConfigMap。kubelet在本Node上通过--manifest-url或--config自动创建的静态Pod将无法引用ConfigMap。
◎ 在Pod对ConfigMap进行挂载(volumeMount)操作时,在容器内部只能挂载为“目录”,无法挂载为“文件”。在挂载到容器内部后,在目录下将包含ConfigMap定义的每个item,如果在该目录下原来还有其他文件,则容器内的该目录将被挂载的ConfigMap覆盖。如果应用程序需要保留原来的其他文件,则需要进行额外的处理。可以将ConfigMap挂载到容器内部的临时目录,再通过启动脚本将配置文件复制或者链接到(cp或link命令)应用所用的实际配置目录下。