Issue while running deploy-apis.sh

I am facing some issues while running the deploy-apis.sh
Below is the output
@@@@@@@@@ Bootstrap swarm
 [WARNING]: Found variable using reserved name: hosts

PLAY [swarm-manager] **************************************************************************************************************************
 [WARNING]: While constructing a mapping from /home/kmadmin/sunbird/infosys-devops/ansible/inventories/dev/group_vars/dev, line 2, column 1,
found a duplicate dict key (keycloak_admin_username). Using last defined value only.

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY [swarm-manager] **************************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

TASK [bootstrap_swarm : Install jq] ***********************************************************************************************************
ok: [localhost]

TASK [bootstrap_swarm : Create networks] ******************************************************************************************************
failed: [localhost] (item=logger) => {"changed": true, "cmd": "docker network create --driver overlay logger", "delta": "0:00:01.540994", "end": "2017-11-10 19:25:45.045149", "failed": true, "item": "logger", "msg": "non-zero return code", "rc": 1, "start": "2017-11-10 19:25:43.504155", "stderr": "Error response from daemon: network with name logger already exists", "stderr_lines": ["Error response from daemon: network with name logger already exists"], "stdout": "", "stdout_lines": []}
failed: [localhost] (item=application_default) => {"changed": true, "cmd": "docker network create --driver overlay application_default", "delta": "0:00:00.211050", "end": "2017-11-10 19:25:45.385517", "failed": true, "item": "application_default", "msg": "non-zero return code", "rc": 1, "start": "2017-11-10 19:25:45.174467", "stderr": "Error response from daemon: network with name application_default already exists", "stderr_lines": ["Error response from daemon: network with name application_default already exists"], "stdout": "", "stdout_lines": []}
failed: [localhost] (item=api-manager_default) => {"changed": true, "cmd": "docker network create --driver overlay api-manager_default", "delta": "0:00:00.013652", "end": "2017-11-10 19:25:45.534556", "failed": true, "item": "api-manager_default", "msg": "non-zero return code", "rc": 1, "start": "2017-11-10 19:25:45.520904", "stderr": "Error response from daemon: network with name api-manager_default already exists", "stderr_lines": ["Error response from daemon: network with name api-manager_default already exists"], "stdout": "", "stdout_lines": []}
failed: [localhost] (item=monitoring) => {"changed": true, "cmd": "docker network create --driver overlay monitoring", "delta": "0:00:00.017299", "end": "2017-11-10 19:25:45.676876", "failed": true, "item": "monitoring", "msg": "non-zero return code", "rc": 1, "start": "2017-11-10 19:25:45.659577", "stderr": "Error response from daemon: network with name monitoring already exists", "stderr_lines": ["Error response from daemon: network with name monitoring already exists"], "stdout": "", "stdout_lines": []}
failed: [localhost] (item=jenkins_default) => {"changed": true, "cmd": "docker network create --driver overlay jenkins_default", "delta": "0:00:00.014517", "end": "2017-11-10 19:25:45.824681", "failed": true, "item": "jenkins_default", "msg": "non-zero return code", "rc": 1, "start": "2017-11-10 19:25:45.810164", "stderr": "Error response from daemon: network with name jenkins_default already exists", "stderr_lines": ["Error response from daemon: network with name jenkins_default already exists"], "stdout": "", "stdout_lines": []}
failed: [localhost] (item=proxy_default) => {"changed": true, "cmd": "docker network create --driver overlay proxy_default", "delta": "0:00:00.016246", "end": "2017-11-10 19:25:45.969586", "failed": true, "item": "proxy_default", "msg": "non-zero return code", "rc": 1, "start": "2017-11-10 19:25:45.953340", "stderr": "Error response from daemon: network with name proxy_default already exists", "stderr_lines": ["Error response from daemon: network with name proxy_default already exists"], "stdout": "", "stdout_lines": []}
...ignoring

PLAY RECAP ************************************************************************************************************************************
localhost                  : ok=4    changed=1    unreachable=0    failed=0

@@@@@@@@@ Deploy API Manager
[DEPRECATION WARNING]: The use of 'include' for tasks has been deprecated. Use 'import_tasks' for static inclusions or 'include_tasks' for
dynamic inclusions. This feature will be removed in a future release. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
[DEPRECATION WARNING]: include is kept for backwards compatibility but usage is discouraged. The module documentation details page may explain
 more about this rationale.. This feature will be removed in a future release. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
 [WARNING]: While constructing a mapping from /home/kmadmin/sunbird/sunbird-devops/ansible/roles/stack-sunbird/defaults/main.yml, line 2,
column 1, found a duplicate dict key (sunbird_env_logo_url). Using last defined value only.

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************
 [WARNING]: While constructing a mapping from /home/kmadmin/sunbird/infosys-devops/ansible/inventories/dev/group_vars/dev, line 2, column 1,
found a duplicate dict key (keycloak_admin_username). Using last defined value only.

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

TASK [stack-api-manager : Ensure env directory exists] ****************************************************************************************
ok: [localhost]

TASK [stack-api-manager : Ensure stack directory exists] **************************************************************************************
ok: [localhost]

TASK [stack-api-manager : Save configurations into an env file] *******************************************************************************
ok: [localhost]

TASK [stack-api-manager : Save stack file] ****************************************************************************************************
ok: [localhost]

TASK [stack-api-manager : Deploy stack] *******************************************************************************************************
changed: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY RECAP ************************************************************************************************************************************
localhost                  : ok=12   changed=1    unreachable=0    failed=0

@@@@@@@@@ Deploy Admin Utils API
[DEPRECATION WARNING]: The use of 'include' for tasks has been deprecated. Use 'import_tasks' for static inclusions or 'include_tasks' for
dynamic inclusions. This feature will be removed in a future release. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
[DEPRECATION WARNING]: include is kept for backwards compatibility but usage is discouraged. The module documentation details page may explain
 more about this rationale.. This feature will be removed in a future release. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
 [WARNING]: While constructing a mapping from /home/kmadmin/sunbird/sunbird-devops/ansible/roles/stack-sunbird/defaults/main.yml, line 2,
column 1, found a duplicate dict key (sunbird_env_logo_url). Using last defined value only.

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************
 [WARNING]: While constructing a mapping from /home/kmadmin/sunbird/infosys-devops/ansible/inventories/dev/group_vars/dev, line 2, column 1,
found a duplicate dict key (keycloak_admin_username). Using last defined value only.

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

TASK [stack-adminutil : Ensure env directory exists] ******************************************************************************************
ok: [localhost]

TASK [stack-adminutil : Ensure stack directory exists] ****************************************************************************************
ok: [localhost]

TASK [stack-adminutil : Save stack file] ******************************************************************************************************
ok: [localhost]

TASK [stack-adminutil : Deploy stack] *********************************************************************************************************
changed: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY RECAP ************************************************************************************************************************************
localhost                  : ok=11   changed=1    unreachable=0    failed=0

@@@@@@@@@ Onboard APIs

PLAY [Kong API] *******************************************************************************************************************************
 [WARNING]: While constructing a mapping from /home/kmadmin/sunbird/infosys-devops/ansible/inventories/dev/group_vars/dev, line 2, column 1,
found a duplicate dict key (keycloak_admin_username). Using last defined value only.

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

TASK [kong-api : Ensure pip is installed] *****************************************************************************************************
ok: [localhost]

TASK [kong-api : Ensure retry is installed] ***************************************************************************************************
ok: [localhost]

TASK [kong-api : Save api details to json file] ***********************************************************************************************
ok: [localhost]

TASK [kong-api : Copy kong api scripts] *******************************************************************************************************
ok: [localhost]

TASK [kong-api : Ensure kong_admin_api_url is available before running the scripts] ***********************************************************
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (60 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (59 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (58 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (57 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (56 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (55 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (54 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (53 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (52 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (51 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (50 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (49 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (48 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (47 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (46 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (45 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (44 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (43 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (42 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (41 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (40 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (39 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (38 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (37 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (36 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (35 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (34 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (33 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (32 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (31 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (30 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (29 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (28 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (27 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (26 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (25 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (24 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (23 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (22 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (21 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (20 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (19 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (18 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (17 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (16 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (15 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (14 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (13 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (12 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (11 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (10 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (9 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (8 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (7 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (6 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (5 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (4 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (3 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (2 retries left).
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (1 retries left).
fatal: [localhost]: FAILED! => {"attempts": 60, "changed": false, "content": "", "failed": true, "msg": "Status code was not [200]: Request failed: <urlopen error [Errno 111] Connection refused>", "redirected": false, "status": -1, "url": "http://<IP_ADDRESS>:8001/status"}
	to retry, use: --limit @/home/kmadmin/sunbird/sunbird-devops/ansible/api-manager.retry

PLAY RECAP ************************************************************************************************************************************
localhost                  : ok=5    changed=0    unreachable=0    failed=1

Please help, the TASK [bootstrap_swarm : Create networks] had ran successfully on the first run, but the TASK [kong-api : Ensure kong_admin_api_url is available before running the scripts]  is creating a problem. Should kong_admin_api_url be already set up ? Is there a place to configure it ?

The error shows connection error to kong url http://<IP_ADDRESS>:8001/status. Please check if url http://<IP_ADDRESS>:8001/status is accessible
If not, check if api-manager_kong service is running. Please follow steps in https://github.com/project-sunbird/sunbird-devops/wiki/Docker-swarm-management-commands#list-services-running-in-swarm to find more details

I did not install or start this service. Is this installed by install-deps.sh script ?

It is installed as part of deploy-apis.sh
On Sat, Nov 11, 2017 at 1:16 PM, Abhishek Govula<EMAIL_ADDRESS>wrote:

I did not install or start this service. Is this installed by
install-deps.sh script ?
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/project-sunbird/sunbird-commons/issues/111#issuecomment-343647458,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAS9UAFmGyT13tJe89AlQgqG5W468naAks5s1VC4gaJpZM4QZnqH
.

--
Regards,
Deepak N
@endeepak

@endeepak the log shows that the docker network failed to be created. is that expected behaviour?

@kochhar The network creation error can be ignored while running script second time onwards or running multiple scripts which try creating the same set of networks. In general any ansible output which has failed followed by ignoring... line fall into this category
Networks can be checked via
docker network ls

@endeepak The service is running in the docker swarm, but I'm unable to access the url http://<IP_ADDRESS>:8001/status
Below is the output when I try to access the url
# wget http://<IP_ADDRESS>:8001/status
--2017-11-13 11:35:08--  http://<IP_ADDRESS>:8001/status
Connecting to <IP_ADDRESS>:8001... failed: Connection refused.

Output for docker service ls (In the documentation, it is mentioned docker services ls, services is not found in the commands list, I guess this is a spelling error, I tried docker service ls )
# docker service ls
ID                  NAME                  MODE                REPLICAS            IMAGE                                   PORTS
gnc39e4c4qaq        adminutil_adminutil   replicated          1/1                 sunbird/adminutil:0.0.1-SNAPSHOT-gold   *:4000->4000/tcp
vi50w0snc8o4        api-manager_echo      replicated          1/1                 sunbird/echo-server:0.0.2-silver        *:9595->9595/tcp
z55ymej2b0j5        api-manager_kong      replicated          0/1                 sunbird/kong:0.9.9                      *:8000->8000/tcp,*:8001->8001/tcp

Below is the output for docker network ls
NETWORK ID          NAME                  DRIVER              SCOPE
o1r77vxhymtd        api-manager_default   overlay             swarm
btww7a3r2im8        application_default   overlay             swarm
7832409641ed        bridge                bridge              local
28bff6631190        docker_gwbridge       bridge              local
718fc03f95b9        host                  host                local
9lvcf0i23w53        ingress               overlay             swarm
ncks03iw1wrb        jenkins_default       overlay             swarm
hqa8zofem4vm        logger                overlay             swarm
17gb2b06kgz5        monitoring            overlay             swarm
852c0fac25c7        none                  null                local
vf7fmsr0hhsk        proxy_default         overlay             swarm

Please help me for further steps, I tried restarting the services
# docker service update api-manager_kong --force
api-manager_kong
Since --detach=false was not specified, tasks will be updated in the background.
In a future release, --detach=false will become the default.

#docker service ls
ID                  NAME                  MODE                REPLICAS            IMAGE                                   PORTS
gnc39e4c4qaq        adminutil_adminutil   replicated          1/1                 sunbird/adminutil:0.0.1-SNAPSHOT-gold   *:4000->4000/tcp
vi50w0snc8o4        api-manager_echo      replicated          1/1                 sunbird/echo-server:0.0.2-silver        *:9595->9595/tcp
z55ymej2b0j5        api-manager_kong      replicated          0/1                 sunbird/kong:0.9.9                      *:8000->8000/tcp,*:8001->8001/tcp

But I had no success accessing the URL http://<IP_ADDRESS>:8001/status
Please help me for further steps

@endeepak
Please find the output for the docker ps
First run
# docker ps
CONTAINER ID        IMAGE                                   COMMAND                  CREATED             STATUS                           PORTS                               NAMES
40dd566be180        sunbird/kong:0.9.9                      "/docker-entrypoin..."   11 seconds ago      Up 1 second (health: starting)   7946/tcp, 8000-8001/tcp, 8443/tcp   api-manager_kong.1.as4dhge2rxtzsqb4r7mbh5lrq
c83ee7bc98da        sunbird/adminutil:0.0.1-SNAPSHOT-gold   "/bin/sh -c '/usr/..."   32 minutes ago      Up 32 minutes (healthy)          4000/tcp                            adminutil_adminutil.1.79e18x5vjkly63esgthaw8wc0
464f505333d0        sunbird/echo-server:0.0.2-silver        "python server.py"       2 hours ago         Up 2 hours (healthy)             9595/tcp                            api-manager_echo.1.pyl7ocuna2g07s23hal9wft72

Second run
# docker ps
CONTAINER ID        IMAGE                                   COMMAND                  CREATED             STATUS                    PORTS               NAMES
c83ee7bc98da        sunbird/adminutil:0.0.1-SNAPSHOT-gold   "/bin/sh -c '/usr/..."   32 minutes ago      Up 32 minutes (healthy)   4000/tcp            adminutil_adminutil.1.79e18x5vjkly63esgthaw8wc0
464f505333d0        sunbird/echo-server:0.0.2-silver        "python server.py"       2 hours ago         Up 2 hours (healthy)      9595/tcp            api-manager_echo.1.pyl7ocuna2g07s23hal9wft72

Output for docker images
# docker images
REPOSITORY            TAG                 IMAGE ID            CREATED             SIZE
sunbird/adminutil     <none>              8c21fbe29395        7 days ago          96.9MB
sunbird/echo-server   <none>              a36ad562df0d        7 weeks ago         72MB
sunbird/kong          <none>              7f651085ff75        7 weeks ago         433MB

@Abhishek-Govula The line api-manager_kong      replicated          0/1 indicates 0 out of 1 replica is running. This means service hasn't started. Since you have setup postgres manually you might not have added database users  needed for API manager
Could you please get service info by running https://github.com/project-sunbird/sunbird-devops/wiki/Docker-swarm-management-commands#get-info-of-service-in-swarm

@endeepak Can you please share the link to the scripts to create the users required ?
Output for info of services
# docker service ls | grep api-manager_kong
z55ymej2b0j5        api-manager_kong      replicated          0/1                 sunbird/kong:0.9.9                      *:8000->8000/tcp,*:8001->8001/tcp

#docker service ps api-manager_kong --no-trunc
ID                          NAME                     IMAGE                                                                                NODE                DESIRED STATE       CURRENT STATE            ERROR                       PORTS
dipy41ncdo9xux73mmq6ykzaa   api-manager_kong.1       sunbird/kong:0.9.9@sha256:ab348bf41fcd62ec3e31709f04072982546a7151c0a025cc87e906965b69e98b   kmadmin             Running             Starting 4 seconds ago
b2277n1ds3a753zdo82xk497d    \_ api-manager_kong.1   sunbird/kong:0.9.9@sha256:ab348bf41fcd62ec3e31709f04072982546a7151c0a025cc87e906965b69e98b   kmadmin             Shutdown            Failed 9 seconds ago     "task: non-zero exit (1)"
lit54oxtg74xflgjuoi9lmjon    \_ api-manager_kong.1   sunbird/kong:0.9.9@sha256:ab348bf41fcd62ec3e31709f04072982546a7151c0a025cc87e906965b69e98b   kmadmin             Shutdown            Failed 22 seconds ago    "task: non-zero exit (1)"
6kesuabb10d7aj35o25jvyie4    \_ api-manager_kong.1   sunbird/kong:0.9.9@sha256:ab348bf41fcd62ec3e31709f04072982546a7151c0a025cc87e906965b69e98b   kmadmin             Shutdown            Failed 33 seconds ago    "task: non-zero exit (1)"
hftabkia8ahgifen8kxld4f93    \_ api-manager_kong.1   sunbird/kong:0.9.9@sha256:ab348bf41fcd62ec3e31709f04072982546a7151c0a025cc87e906965b69e98b   kmadmin             Shutdown            Failed 45 seconds ago    "task: non-zero exit (1)"

# docker service logs api-manager_kong --tail 200 -f
api-manager_kong.1.k8xqg03qhrk9@kmadmin    | KONG_CLUSTER_LISTEN: <IP_ADDRESS>:7946
api-manager_kong.1.lkgm5iwv2770@kmadmin    | KONG_CLUSTER_LISTEN: <IP_ADDRESS>:7946
api-manager_kong.1.b2277n1ds3a7@kmadmin    | KONG_CLUSTER_LISTEN: <IP_ADDRESS>:7946
api-manager_kong.1.dipy41ncdo9x@kmadmin    | KONG_CLUSTER_LISTEN: <IP_ADDRESS>:7946
api-manager_kong.1.dipy41ncdo9x@kmadmin    | Error: /usr/local/share/lua/5.1/kong/cmd/start.lua:18: [postgres error] connection refused
api-manager_kong.1.lkgm5iwv2770@kmadmin    | Error: /usr/local/share/lua/5.1/kong/cmd/start.lua:18: [postgres error] connection refused
api-manager_kong.1.b2277n1ds3a7@kmadmin    | Error: /usr/local/share/lua/5.1/kong/cmd/start.lua:18: [postgres error] connection refused
api-manager_kong.1.k8xqg03qhrk9@kmadmin    | Error: /usr/local/share/lua/5.1/kong/cmd/start.lua:18: [postgres error] connection refused
api-manager_kong.1.dipy41ncdo9x@kmadmin    |
api-manager_kong.1.k8xqg03qhrk9@kmadmin    |
api-manager_kong.1.b2277n1ds3a7@kmadmin    |
api-manager_kong.1.lkgm5iwv2770@kmadmin    |
api-manager_kong.1.b2277n1ds3a7@kmadmin    |   Run with --v (verbose) or --vv (debug) for more details
api-manager_kong.1.lkgm5iwv2770@kmadmin    |   Run with --v (verbose) or --vv (debug) for more details
api-manager_kong.1.dipy41ncdo9x@kmadmin    |   Run with --v (verbose) or --vv (debug) for more details
api-manager_kong.1.k8xqg03qhrk9@kmadmin    |   Run with --v (verbose) or --vv (debug) for more details
api-manager_kong.1.2ngrimenq02b@kmadmin    | KONG_CLUSTER_LISTEN: <IP_ADDRESS>:7946
api-manager_kong.1.2ngrimenq02b@kmadmin    | Error: /usr/local/share/lua/5.1/kong/cmd/start.lua:18: [postgres error] connection refused
api-manager_kong.1.2ngrimenq02b@kmadmin    |
api-manager_kong.1.2ngrimenq02b@kmadmin    |   Run with --v (verbose) or --vv (debug) for more details
api-manager_kong.1.v6d91hykfynu@kmadmin    | KONG_CLUSTER_LISTEN: <IP_ADDRESS>:7946
api-manager_kong.1.v6d91hykfynu@kmadmin    | Error: /usr/local/share/lua/5.1/kong/cmd/start.lua:18: [postgres error] connection refused

Output for the databases and the tables for user quartz in postgres
quartz=> \l
                             List of databases
   Name    |  Owner   | Encoding | Collate | Ctype |   Access privileges
-----------+----------+----------+---------+-------+-----------------------
 postgres  | postgres | UTF8     | en_IN   | en_IN |
 quartz    | quartz   | UTF8     | en_IN   | en_IN |
 template0 | postgres | UTF8     | en_IN   | en_IN | =c/postgres          +
           |          |          |         |       | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_IN   | en_IN | =c/postgres          +
           |          |          |         |       | postgres=CTc/postgres
(4 rows)

quartz=> \dt
                 List of relations
 Schema |           Name           | Type  | Owner
--------+--------------------------+-------+--------
 public | qrtz_blob_triggers       | table | quartz
 public | qrtz_calendars           | table | quartz
 public | qrtz_cron_triggers       | table | quartz
 public | qrtz_fired_triggers      | table | quartz
 public | qrtz_job_details         | table | quartz
 public | qrtz_locks               | table | quartz
 public | qrtz_paused_trigger_grps | table | quartz
 public | qrtz_scheduler_state     | table | quartz
 public | qrtz_simple_triggers     | table | quartz
 public | qrtz_simprop_triggers    | table | quartz
 public | qrtz_triggers            | table | quartz
(11 rows)

@Abhishek-Govula If you are setting up postgres on new server, we recommend you to use install-db.sh as mentioned in docs. That would setup users and databases required for all services used in sunbird
If you prefer to setup database and users manually,
Create databases listed in:
https://github.com/project-sunbird/sunbird-devops/blob/f8000e3d66a3983673bde2793c92313e3d725b4e/ansible/inventories/sample/group_vars/postgresql-master#L1
Create users with permission mentioned in below link:
https://github.com/project-sunbird/sunbird-devops/blob/f8000e3d66a3983673bde2793c92313e3d725b4e/ansible/inventories/sample/group_vars/sample#L131
As you might have noticed, we use ansible for setting up infrastructure. The role for creating postgres users and databases is https://github.com/project-sunbird/sunbird-devops/tree/master/ansible/roles/postgresql

@endeepak I tried using the install-dbs.sh script to install the postgres, but the service was not starting up for the installation done by the script which made me go for the manual installation.
Now I have set up the databases required
postgresql_databases:
  - name: "{{kong_postgres_database}}"
  - name: "{{keycloak_postgres_database}}"
  - name: "{{application_postgres_database}}"

Below is the output for the list of databases, I have given all the permissions to the respective users as well
postgres=# \l
                                          List of databases
      Name       |      Owner      | Encoding | Collate | Ctype |          Access privileges
-----------------+-----------------+----------+---------+-------+-------------------------------------
 api_manager_dev | api_manager_dev | UTF8     | en_IN   | en_IN | =Tc/api_manager_dev                +
                 |                 |          |         |       | api_manager_dev=CTc/api_manager_dev
 keycloak        | keycloak        | UTF8     | en_IN   | en_IN | =Tc/keycloak                       +
                 |                 |          |         |       | keycloak=CTc/keycloak
 postgres        | postgres        | UTF8     | en_IN   | en_IN |
 quartz          | quartz          | UTF8     | en_IN   | en_IN | =Tc/quartz                         +
                 |                 |          |         |       | quartz=CTc/quartz
 template0       | postgres        | UTF8     | en_IN   | en_IN | =c/postgres                        +
                 |                 |          |         |       | postgres=CTc/postgres
 template1       | postgres        | UTF8     | en_IN   | en_IN | =c/postgres                        +
                 |                 |          |         |       | postgres=CTc/postgres
(6 rows)

Please let me know if this is sufficient, the deploy-apis.sh is still resulting in the same issue
Also if you can let me know if there is any way to run this users and role creation script alone, so that the user and roles can be created as per the existing script. It will be helpful. since the postgres installed by the install-dbs.sh was not running on my system, if I can run the users script alone, it would work for me I suppose

@Abhishek-Govula The databases and users seems to correct. Can you please try accessing the postgres db from app server using postgres client. This will help in debugging any network or firewall related issues.
Please ensure postgres_hba.conf  has the postgresql_hba_entries listed in https://github.com/project-sunbird/sunbird-devops/blob/f8000e3d66a3983673bde2793c92313e3d725b4e/ansible/inventories/sample/group_vars/sample#L131.  You can read more about postgres_hba.conf in https://www.postgresql.org/docs/9.5/static/auth-pg-hba-conf.html
Please raise an issue for postgres installation error. This would help us in fixing any issues with that script

@endeepak Right now, I am running both the dbs and application on a single server since I have firewall restrictions. Will this cause a problem ?
Also, do I need to add all the entries, like
## PostgreSQL config
postgresql_hba_entries:
  - { type: local, database: all, user: postgres, auth_method: peer }
  - { type: local, database: quartz, user: quartz, auth_method: password }
  - { type: local, database: keycloak, user: keycloak, auth_method: password }
  - { type: local, database: api_manager_dev, user: api_manager_dev, auth_method: password }
  - { type: local, database: all, user: all, auth_method: peer }
  - { type: host, database: all, user: all, address: '<IP_ADDRESS>/32', auth_method: md5 }
  - { type: host, database: all, user: all, address: '::1/128', auth_method: md5 }
  - { type: host, database: all, user: all, address: '{{ swarm_address_space }}', auth_method: md5 }

I am assuming that the auth method is password since this is being picked up from the configuration yml file. Are these entries correct ?

I guess postgres only allows peer authentication by default. Can you please
provide existing entries in postgres_hba.conf
On Nov 13, 2017 2:48 PM, "Abhishek Govula"<EMAIL_ADDRESS>wrote:

@endeepak https://github.com/endeepak Right now, I am running both the
dbs and application on a single server since I have firewall restrictions.
Will this cause a problem ?
Also, do I need to add all the entries, like
PostgreSQL config
postgresql_hba_entries:

{ type: local, database: all, user: postgres, auth_method: peer }
{ type: local, database: quartz, user: quartz, auth_method: password }
{ type: local, database: keycloak, user: keycloak, auth_method: password }
{ type: local, database: api_manager_dev, user: api_manager_dev, auth_method: password }
{ type: local, database: all, user: all, auth_method: peer }
{ type: host, database: all, user: all, address: '<IP_ADDRESS>/32', auth_method: md5 }
{ type: host, database: all, user: all, address: '::1/128', auth_method: md5 }
{ type: host, database: all, user: all, address: '{{ swarm_address_space }}', auth_method: md5 }

I am assuming that the auth method is password since this is being picked
up from the configuration yml file. Are these entries correct ?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/project-sunbird/sunbird-commons/issues/111#issuecomment-343858473,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAS9UH5wEdFI74bQ3jQbIY8-TSUZ0VHQks5s2Al1gaJpZM4QZnqH
.

@endeepak  Tried all the possible combinations with database, auth_method and user, but still no success.
Can I run the init-dbs.sh, Since postgres is already installed, I am assuming the check is done, to see whether the database server is installed and then the roles are created.

Hi Abhishek, the scripts are idempotent so they will bring the system to
the desired state. You can run them again.
On Mon, Nov 13, 2017 at 4:21 PM Abhishek Govula<EMAIL_ADDRESS>wrote:

@endeepak https://github.com/endeepak Tried all the possible
combinations with database, auth_method and user, but still no success.
Can I run the init-dbs.sh, Since postgres is already installed, I am
assuming the check is done, to see whether the database server is installed
and then the roles are created.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/project-sunbird/sunbird-commons/issues/111#issuecomment-343882788,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAGVqq2oRMzZX2xgxclLoIjdlDysNnDHks5s2B8hgaJpZM4QZnqH
.

@kochhar I don't think they are idempotent, I had ran the scripts and my current setup of postgres has broken. I had to reinstall and create all the users and roles again :(

@kochhar This is happening because of the hba configuration of postgres. Below is my configuration
postgresql_hba_entries:
  - { type: local, database: all, user: postgres, auth_method: peer }
  - { type: local, database: all, user: quartz, auth_method: peer }
  - { type: local, database: all, user: api_manager_dev, auth_method: peer }
  - { type: local, database: all, user: keycloak, auth_method: peer }
  - { type: local, database: all, user: all, auth_method: peer }
  - { type: host, database: all, user: all, address: '<IP_ADDRESS>/32', auth_method: md5 }
  - { type: host, database: all, user: all, address: '::1/128', auth_method: md5 }
  - { type: host, database: all, user: all, address: '{{ swarm_address_space }}', auth_method: md5 }

Fresh instance of postgres which is running fine, fails to connect after the configuration is modified by the install-dbs.sh script. Can someone please tell me if this configuration is correct ?
Below is the configuration of postgres at /etc/postgresql/9.5/main/pg_hba.conf after I run the install-dbs.sh
#
# Ansible managed
#
# PostgreSQL Client Authentication Configuration File
# ===================================================
#
# See: https://www.postgresql.org/docs/current/static/auth-pg-hba-conf.html

local all postgres    peer
local all quartz    peer
local all api_manager_dev    peer
local all keycloak    peer
local all all    peer
host all all <IP_ADDRESS>/32   md5
host all all ::1/128   md5
host all all    md5

After this configuration is being setup by the script, psql command is failing and the port on which postgres is running is also not being displayed in netstat -ntlpu

You need to fill the swarm_address_space variable in the inventory
group_vars
On Nov 13, 2017 6:56 PM, "Abhishek Govula"<EMAIL_ADDRESS>wrote:

@kochhar https://github.com/kochhar This is happening because of the
hba configuration of postgres. Below is my configuration
postgresql_hba_entries:

{ type: local, database: all, user: postgres, auth_method: peer }
{ type: local, database: all, user: quartz, auth_method: peer }
{ type: local, database: all, user: api_manager_dev, auth_method: peer }
{ type: local, database: all, user: keycloak, auth_method: peer }
{ type: local, database: all, user: all, auth_method: peer }
{ type: host, database: all, user: all, address: '<IP_ADDRESS>/32', auth_method: md5 }
{ type: host, database: all, user: all, address: '::1/128', auth_method: md5 }
{ type: host, database: all, user: all, address: '{{ swarm_address_space }}', auth_method: md5 }

Fresh instance of postgres which is running fine, fails to connect after
the configuration is modified by the install-dbs.sh script. Can someone
please tell me if this configuration is correct ?
Below is the configuration of postgres at /etc/postgresql/9.5/main/pg_
hba.conf after I run the install-dbs.sh

Ansible managed

PostgreSQL Client Authentication Configuration File
===================================================

See: https://www.postgresql.org/docs/current/static/auth-pg-hba-conf.html
local all postgres    peer
local all quartz    peer
local all api_manager_dev    peer
local all keycloak    peer
local all all    peer
host all all <IP_ADDRESS>/32   md5
host all all ::1/128   md5
host all all    md5
After this configuration is being setup by the script, psql command is
failing and the port on which postgres is running is also not being
displayed in netstat -ntlpu
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/project-sunbird/sunbird-commons/issues/111#issuecomment-343918197,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAS9UAPCA6H_un3yCk-dc2ufP3JngCMFks5s2EOKgaJpZM4QZnqH
.

@endeepak Right now, I am running both app and db on the same server, so should this be the internal ip addr ?

It needs to subnet address in cidr format
On Nov 13, 2017 7:20 PM, "Abhishek Govula"<EMAIL_ADDRESS>wrote:

@endeepak https://github.com/endeepak Right now, I am running both app
and db on the same server, so should this be the internal ip addr ?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/project-sunbird/sunbird-commons/issues/111#issuecomment-343924297,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAS9UMkszi3T3Z99_KyzsvUN8ubCztGQks5s2EkxgaJpZM4QZnqH
.

@endeepak Is there a way I can know this address before configuration, or can this be any subnet adress ?

@Abhishek-Govula If it is single server, please use <ip-address>/32  for now.

@Abhishek-Govula If you plan to multiple servers in future, please refer to https://networkengineering.stackexchange.com/questions/7106/how-do-you-calculate-the-prefix-network-subnet-and-host-numbers for finding network address in CIDR format

I have enabled all incoming connections from all the ports on Postgres using
host    all             all             <IP_ADDRESS>/0            md5

This should enable the api-manager_kong app to access the Postgres, yet I'm facing the issue. I think the way api-manager_kong trying to connect to the database is causing the issue. Is there a way to replicate the running of this docker image as it is run by the docker service?

Please try command for testing postgres connection
psql -H <db-ip> -U api_manager_dev -P
# Enter the password mentioned as kong_postgres_password in inventory group vars

Please provide logs from kong service

$ psql -H <IP_ADDRESS> -U api_manager_dev -P
/usr/lib/postgresql/9.5/bin/psql: option requires an argument -- 'P'
Try "psql --help" for more information.

Update on the kong service
The api-manger_kong is running now.
# docker service ls
ID                  NAME                  MODE                REPLICAS            IMAGE                                   PORTS
gnc39e4c4qaq        adminutil_adminutil   replicated          1/1                 sunbird/adminutil:0.0.1-SNAPSHOT-gold   *:4000->4000/tcp
vi50w0snc8o4        api-manager_echo      replicated          1/1                 sunbird/echo-server:0.0.2-silver        *:9595->9595/tcp
z55ymej2b0j5        api-manager_kong      replicated          1/1                 sunbird/kong:0.9.9                      *:8000->8000/tcp,*:8001->8001/tcp

Logs
# docker ps
CONTAINER ID        IMAGE                                   COMMAND                  CREATED             STATUS                    PORTS                               NAMES
845d930ab72b        sunbird/kong:0.9.9                      "/docker-entrypoin..."   34 minutes ago      Up 33 minutes (healthy)   7946/tcp, 8000-8001/tcp, 8443/tcp   api-manager_kong.1.drcf3qhtw388bizz9lykaneqn
8440870f4bec        sunbird/adminutil:0.0.1-SNAPSHOT-gold   "/bin/sh -c '/usr/..."   35 minutes ago      Up 34 minutes (healthy)   4000/tcp                            adminutil_adminutil.1.crunka9z4n4odrfo5boblbqwm
a53d084a61fc        sunbird/echo-server:0.0.2-silver        "python server.py"       5 hours ago         Up 5 hours (healthy)      9595/tcp                            api-manager_echo.1.y1zbteq59cljxz0rm138siwvb

But the connection error
FAILED - RETRYING: Ensure kong_admin_api_url is available before running the scripts (1 retries left).
fatal: [localhost]: FAILED! => {"attempts": 60, "changed": false, "content": "", "failed": true, "msg": "Status code was not [200]: Request failed: <urlopen error [Errno 111] Connection refused>", "redirected": false, "status": -1, "url": "http://<IP_ADDRESS>:8001/status"}
	to retry, use: --limit @/home/kmadmin/sunbird/sunbird-devops/ansible/api-manager.retry

PLAY RECAP ************************************************************************************************************************************
localhost                  : ok=5    changed=0    unreachable=0    failed=1

still persists. Please help !

Please try to curl the url on command line
curl http://<IP_ADDRESS>:8001/status
curl http://<IP_ADDRESS>:8000

Please check if private ip of the app server is <IP_ADDRESS>. If not, please set the the value of kong_host in inventory group_vars with the correct ip address

App has started up successfully, but with few warnings. Below is the output when I ran deploy-apis.sh
@@@@@@@@@ Bootstrap swarm
 [WARNING]: Found variable using reserved name: hosts

PLAY [swarm-manager] **************************************************************************************************************************
 [WARNING]: While constructing a mapping from /home/kmadmin/sunbird/infosys-devops/ansible/inventories/dev/group_vars/dev, line 2, column 1,
found a duplicate dict key (keycloak_admin_username). Using last defined value only.

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY [swarm-manager] **************************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

TASK [bootstrap_swarm : Install jq] ***********************************************************************************************************
ok: [localhost]

TASK [bootstrap_swarm : Create networks] ******************************************************************************************************
failed: [localhost] (item=logger) => {"changed": true, "cmd": "docker network create --driver overlay logger", "delta": "0:00:00.415227", "end": "2017-11-14 09:26:03.925911", "failed": true, "item": "logger", "msg": "non-zero return code", "rc": 1, "start": "2017-11-14 09:26:03.510684", "stderr": "Error response from daemon: network with name logger already exists", "stderr_lines": ["Error response from daemon: network with name logger already exists"], "stdout": "", "stdout_lines": []}
failed: [localhost] (item=application_default) => {"changed": true, "cmd": "docker network create --driver overlay application_default", "delta": "0:00:00.015860", "end": "2017-11-14 09:26:04.073589", "failed": true, "item": "application_default", "msg": "non-zero return code", "rc": 1, "start": "2017-11-14 09:26:04.057729", "stderr": "Error response from daemon: network with name application_default already exists", "stderr_lines": ["Error response from daemon: network with name application_default already exists"], "stdout": "", "stdout_lines": []}
failed: [localhost] (item=api-manager_default) => {"changed": true, "cmd": "docker network create --driver overlay api-manager_default", "delta": "0:00:00.013232", "end": "2017-11-14 09:26:04.223648", "failed": true, "item": "api-manager_default", "msg": "non-zero return code", "rc": 1, "start": "2017-11-14 09:26:04.210416", "stderr": "Error response from daemon: network with name api-manager_default already exists", "stderr_lines": ["Error response from daemon: network with name api-manager_default already exists"], "stdout": "", "stdout_lines": []}
failed: [localhost] (item=monitoring) => {"changed": true, "cmd": "docker network create --driver overlay monitoring", "delta": "0:00:00.018119", "end": "2017-11-14 09:26:04.373542", "failed": true, "item": "monitoring", "msg": "non-zero return code", "rc": 1, "start": "2017-11-14 09:26:04.355423", "stderr": "Error response from daemon: network with name monitoring already exists", "stderr_lines": ["Error response from daemon: network with name monitoring already exists"], "stdout": "", "stdout_lines": []}
failed: [localhost] (item=jenkins_default) => {"changed": true, "cmd": "docker network create --driver overlay jenkins_default", "delta": "0:00:00.012120", "end": "2017-11-14 09:26:04.516943", "failed": true, "item": "jenkins_default", "msg": "non-zero return code", "rc": 1, "start": "2017-11-14 09:26:04.504823", "stderr": "Error response from daemon: network with name jenkins_default already exists", "stderr_lines": ["Error response from daemon: network with name jenkins_default already exists"], "stdout": "", "stdout_lines": []}
failed: [localhost] (item=proxy_default) => {"changed": true, "cmd": "docker network create --driver overlay proxy_default", "delta": "0:00:00.015850", "end": "2017-11-14 09:26:04.669678", "failed": true, "item": "proxy_default", "msg": "non-zero return code", "rc": 1, "start": "2017-11-14 09:26:04.653828", "stderr": "Error response from daemon: network with name proxy_default already exists", "stderr_lines": ["Error response from daemon: network with name proxy_default already exists"], "stdout": "", "stdout_lines": []}
...ignoring

PLAY RECAP ************************************************************************************************************************************
localhost                  : ok=4    changed=1    unreachable=0    failed=0

@@@@@@@@@ Deploy API Manager
[DEPRECATION WARNING]: The use of 'include' for tasks has been deprecated. Use 'import_tasks' for static inclusions or 'include_tasks' for
dynamic inclusions. This feature will be removed in a future release. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
[DEPRECATION WARNING]: include is kept for backwards compatibility but usage is discouraged. The module documentation details page may explain
 more about this rationale.. This feature will be removed in a future release. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
 [WARNING]: While constructing a mapping from /home/kmadmin/sunbird/sunbird-devops/ansible/roles/stack-sunbird/defaults/main.yml, line 2,
column 1, found a duplicate dict key (sunbird_env_logo_url). Using last defined value only.

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************
 [WARNING]: While constructing a mapping from /home/kmadmin/sunbird/infosys-devops/ansible/inventories/dev/group_vars/dev, line 2, column 1,
found a duplicate dict key (keycloak_admin_username). Using last defined value only.

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

TASK [stack-api-manager : Ensure env directory exists] ****************************************************************************************
ok: [localhost]

TASK [stack-api-manager : Ensure stack directory exists] **************************************************************************************
ok: [localhost]

TASK [stack-api-manager : Save configurations into an env file] *******************************************************************************
changed: [localhost]

TASK [stack-api-manager : Save stack file] ****************************************************************************************************
ok: [localhost]

TASK [stack-api-manager : Deploy stack] *******************************************************************************************************
changed: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY RECAP ************************************************************************************************************************************
localhost                  : ok=12   changed=2    unreachable=0    failed=0

@@@@@@@@@ Deploy Admin Utils API
[DEPRECATION WARNING]: The use of 'include' for tasks has been deprecated. Use 'import_tasks' for static inclusions or 'include_tasks' for
dynamic inclusions. This feature will be removed in a future release. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
[DEPRECATION WARNING]: include is kept for backwards compatibility but usage is discouraged. The module documentation details page may explain
 more about this rationale.. This feature will be removed in a future release. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
 [WARNING]: While constructing a mapping from /home/kmadmin/sunbird/sunbird-devops/ansible/roles/stack-sunbird/defaults/main.yml, line 2,
column 1, found a duplicate dict key (sunbird_env_logo_url). Using last defined value only.

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************
 [WARNING]: While constructing a mapping from /home/kmadmin/sunbird/infosys-devops/ansible/inventories/dev/group_vars/dev, line 2, column 1,
found a duplicate dict key (keycloak_admin_username). Using last defined value only.

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

TASK [stack-adminutil : Ensure env directory exists] ******************************************************************************************
ok: [localhost]

TASK [stack-adminutil : Ensure stack directory exists] ****************************************************************************************
ok: [localhost]

TASK [stack-adminutil : Save stack file] ******************************************************************************************************
changed: [localhost]

TASK [stack-adminutil : Deploy stack] *********************************************************************************************************
changed: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY [swarm-bootstrap-manager] ****************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

PLAY RECAP ************************************************************************************************************************************
localhost                  : ok=11   changed=2    unreachable=0    failed=0

@@@@@@@@@ Onboard APIs

PLAY [Kong API] *******************************************************************************************************************************
 [WARNING]: While constructing a mapping from /home/kmadmin/sunbird/infosys-devops/ansible/inventories/dev/group_vars/dev, line 2, column 1,
found a duplicate dict key (keycloak_admin_username). Using last defined value only.

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

TASK [kong-api : Ensure pip is installed] *****************************************************************************************************
ok: [localhost]

TASK [kong-api : Ensure retry is installed] ***************************************************************************************************
ok: [localhost]

TASK [kong-api : Save api details to json file] ***********************************************************************************************
changed: [localhost]

TASK [kong-api : Copy kong api scripts] *******************************************************************************************************
changed: [localhost]

TASK [kong-api : Ensure kong_admin_api_url is available before running the scripts] ***********************************************************
ok: [localhost]

TASK [kong-api : Run script to save apis] *****************************************************************************************************
changed: [localhost]

PLAY RECAP ************************************************************************************************************************************
localhost                  : ok=7    changed=3    unreachable=0    failed=0

@@@@@@@@@ Onboard Consumers
Using /etc/ansible/ansible.cfg as config file

PLAY [Kong API] *******************************************************************************************************************************
 [WARNING]: While constructing a mapping from /home/kmadmin/sunbird/infosys-devops/ansible/inventories/dev/group_vars/dev, line 2, column 1,
found a duplicate dict key (keycloak_admin_username). Using last defined value only.

TASK [Gathering Facts] ************************************************************************************************************************
ok: [localhost]

TASK [kong-consumer : Ensure pip is installed] ************************************************************************************************
ok: [localhost] => {"cache_update_time":<PHONE_NUMBER>, "cache_updated": false, "changed": false, "failed": false}

TASK [kong-consumer : Ensure PyJWT is installed] **********************************************************************************************
changed: [localhost] => {"changed": true, "cmd": "/usr/bin/pip2 install PyJWT", "failed": false, "name": ["PyJWT"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.1, however version 9.0.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.1, however version 9.0.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Collecting PyJWT\n  Downloading PyJWT-1.5.3-py2.py3-none-any.whl\nInstalling collected packages: PyJWT\nSuccessfully installed PyJWT-1.5.3\n", "stdout_lines": ["Collecting PyJWT", "  Downloading PyJWT-1.5.3-py2.py3-none-any.whl", "Installing collected packages: PyJWT", "Successfully installed PyJWT-1.5.3"], "version": null, "virtualenv": null}

TASK [kong-consumer : Ensure retry is installed] **********************************************************************************************
ok: [localhost] => {"changed": false, "cmd": "/usr/bin/pip2 install retry", "failed": false, "name": ["retry"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.1, however version 9.0.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.1, however version 9.0.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): retry in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied (use --upgrade to upgrade): decorator>=3.4.2 in /usr/local/lib/python2.7/dist-packages (from retry)\nRequirement already satisfied (use --upgrade to upgrade): py<2.0.0,>=1.4.26 in /usr/local/lib/python2.7/dist-packages (from retry)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): retry in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied (use --upgrade to upgrade): decorator>=3.4.2 in /usr/local/lib/python2.7/dist-packages (from retry)", "Requirement already satisfied (use --upgrade to upgrade): py<2.0.0,>=1.4.26 in /usr/local/lib/python2.7/dist-packages (from retry)"], "version": null, "virtualenv": null}

TASK [kong-consumer : Save kong_consumers to json file] ***************************************************************************************
changed: [localhost] => {"changed": true, "checksum": "b514b825d46578514aec046082acbce7913eb962", "dest": "/tmp/kong_consumers.json", "failed": false, "gid": 0, "group": "root", "md5sum": "40fecdd64356321b3937a347b1cf3b92", "mode": "0644", "owner": "root", "size": 2271, "src": "/root/.ansible/tmp/ansible-tmp-1510631926.11-195723650240627/source", "state": "file", "uid": 0}

TASK [kong-consumer : Copy kong api scripts] **************************************************************************************************
ok: [localhost] => {"changed": false, "dest": "/tmp/", "failed": false, "src": "/home/kmadmin/sunbird/sunbird-devops/ansible/static-files/kong-api-scripts"}

TASK [kong-consumer : Ensure kong_admin_api_url is available before running the scripts] ******************************************************
ok: [localhost] => {"access_control_allow_origin": "*", "attempts": 1, "changed": false, "connection": "close", "content_type": "application/json; charset=utf-8", "cookies": {}, "date": "Tue, 14 Nov 2017 03:58:48 GMT", "failed": false, "json": {"database": {"acls": 0, "apis": 134, "basicauth_credentials": 0, "consumers": 0, "hmacauth_credentials": 0, "jwt_secrets": 0, "keyauth_credentials": 0, "nodes": 1, "oauth2_authorization_codes": 0, "oauth2_credentials": 0, "oauth2_tokens": 0, "plugins": 804, "ratelimiting_metrics": 0, "response_ratelimiting_metrics": 0}, "server": {"connections_accepted": 1917, "connections_active": 2, "connections_handled": 1917, "connections_reading": 0, "connections_waiting": 1, "connections_writing": 1, "total_requests": 1920}}, "msg": "OK (unknown bytes)", "redirected": false, "server": "kong/0.9.9", "status": 200, "transfer_encoding": "chunked", "url": "http://localhost:8001/status"}

TASK [kong-consumer : Run script to save consumers] *******************************************************************************************
changed: [localhost] => {"changed": true, "cmd": "python /tmp/kong-api-scripts/kong_consumers.py /tmp/kong_consumers.json --kong-admin-api-url=http://localhost:8001", "delta": "0:03:02.653948", "end": "2017-11-14 09:31:51.364398", "failed": false, "rc": 0, "start": "2017-11-14 09:28:48.710450", "stderr": "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 2 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 4 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 8 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 16 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 2 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 4 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 8 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 16 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 2 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 4 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 8 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 16 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 2 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 4 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 8 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 16 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 2 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 4 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 8 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 16 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 2 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 4 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 8 seconds...\nWARNING:retry.api:HTTP Error 404: Not Found, retrying in 16 seconds...", "stderr_lines": ["WARNING:retry.api:HTTP Error 404: Not Found, retrying in 2 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 4 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 8 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 16 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 2 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 4 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 8 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 16 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 2 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 4 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 8 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 16 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 2 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 4 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 8 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 16 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 2 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 4 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 8 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 16 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 2 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 4 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 8 seconds...", "WARNING:retry.api:HTTP Error 404: Not Found, retrying in 16 seconds..."], "stdout": "Adding consumer api-management-test-user\nExisting groups for consumer api-management-test-user : []\nRequired groups for consumer api-management-test-user : [u'appAdmin', u'contentAdmin', u'contentUser', u'courseAdmin', u'courseUser', u'echoUser', u'mobileAdmin', u'mobileApp', u'noteAdmin', u'noteUser', u'orgAdmin', u'orgUser', u'publicUser', u'userAdmin', u'configUser', u'dataAdmin']\nAdding group appAdmin for consumer api-management-test-user\nAdding group contentAdmin for consumer api-management-test-user\nAdding group contentUser for consumer api-management-test-user\nAdding group courseAdmin for consumer api-management-test-user\nAdding group courseUser for consumer api-management-test-user\nAdding group echoUser for consumer api-management-test-user\nAdding group mobileAdmin for consumer api-management-test-user\nAdding group mobileApp for consumer api-management-test-user\nAdding group noteAdmin for consumer api-management-test-user\nAdding group noteUser for consumer api-management-test-user\nAdding group orgAdmin for consumer api-management-test-user\nAdding group orgUser for consumer api-management-test-user\nAdding group publicUser for consumer api-management-test-user\nAdding group userAdmin for consumer api-management-test-user\nAdding group configUser for consumer api-management-test-user\nAdding group dataAdmin for consumer api-management-test-user\nCreating jwt credentials for consumer api-management-test-user\nJWT token for api-management-test-user is : <JWT TOKEN>\nAdding consumer player\nExisting groups for consumer player : []\nRequired groups for consumer player : [u'appAdmin', u'contentAdmin', u'contentUser', u'courseAdmin', u'courseUser', u'echoUser', u'mobileAdmin', u'mobileApp', u'noteAdmin', u'noteUser', u'orgAdmin', u'orgUser', u'publicUser', u'userAdmin', u'configUser', u'dataAdmin']\nAdding group appAdmin for consumer player\nAdding group contentAdmin for consumer player\nAdding group contentUser for consumer player\nAdding group courseAdmin for consumer player\nAdding group courseUser for consumer player\nAdding group echoUser for consumer player\nAdding group mobileAdmin for consumer player\nAdding group mobileApp for consumer player\nAdding group noteAdmin for consumer player\nAdding group noteUser for consumer player\nAdding group orgAdmin for consumer player\nAdding group orgUser for consumer player\nAdding group publicUser for consumer player\nAdding group userAdmin for consumer player\nAdding group configUser for consumer player\nAdding group dataAdmin for consumer player\nCreating jwt credentials for consumer player\nJWT token for player is :<JWT TOKEN>\nAdding rate_limit for consumer player for API createContent\nrate_limit_plugin_data: {u'config.hour': 1000, u'config.limit_by': u'credential', 'name': 'rate-limiting', 'consumer_id': u'b7b092a2-d0a9-4aa6-b843-0a6b20dc954e'}\nAdding rate_limit for consumer player for API updateContent\nrate_limit_plugin_data: {u'config.hour': 10000, u'config.limit_by': u'credential', 'name': 'rate-limiting', 'consumer_id': u'b7b092a2-d0a9-4aa6-b843-0a6b20dc954e'}\nAdding rate_limit for consumer player for API readContent\nrate_limit_plugin_data: {u'config.hour': 100000, u'config.limit_by': u'credential', 'name': 'rate-limiting', 'consumer_id': u'b7b092a2-d0a9-4aa6-b843-0a6b20dc954e'}\nAdding consumer mobile_admin\nExisting groups for consumer mobile_admin : []\nRequired groups for consumer mobile_admin : [u'mobileAdmin']\nAdding group mobileAdmin for consumer mobile_admin\nCreating jwt credentials for consumer mobile_admin\nJWT token for mobile_admin is : <JWT TOKEN>nCredentials for consumer mobile_admin, key: 4e9b2d6d4fba48259ac88354541362b9, secret: d2ce34c6ce8148a08e120be917620459\nAdding consumer mobile_app\nExisting groups for consumer mobile_app : []\nRequired groups for consumer mobile_app : [u'mobileApp']\nAdding group mobileApp for consumer mobile_app\nCreating jwt credentials for consumer mobile_app\nJWT token for mobile_app is : <JWT TOKEN>\nAdding consumer mobile_device\nExisting groups for consumer mobile_device : []\nRequired groups for consumer mobile_device : [u'contentUser', u'courseUser', u'appAdmin', u'configUser', u'publicUser', u'userAdmin', u'contentAdmin']\nAdding group contentUser for consumer mobile_device\nAdding group courseUser for consumer mobile_device\nAdding group appAdmin for consumer mobile_device\nAdding group configUser for consumer mobile_device\nAdding group publicUser for consumer mobile_device\nAdding group userAdmin for consumer mobile_device\nAdding group contentAdmin for consumer mobile_device\nCreating jwt credentials for consumer mobile_device\nJWT token for mobile_device is : <JWT TOKEN>", "stdout_lines": ["Adding consumer api-management-test-user", "Existing groups for consumer api-management-test-user : []", "Required groups for consumer api-management-test-user : [u'appAdmin', u'contentAdmin', u'contentUser', u'courseAdmin', u'courseUser', u'echoUser', u'mobileAdmin', u'mobileApp', u'noteAdmin', u'noteUser', u'orgAdmin', u'orgUser', u'publicUser', u'userAdmin', u'configUser', u'dataAdmin']", "Adding group appAdmin for consumer api-management-test-user", "Adding group contentAdmin for consumer api-management-test-user", "Adding group contentUser for consumer api-management-test-user", "Adding group courseAdmin for consumer api-management-test-user", "Adding group courseUser for consumer api-management-test-user", "Adding group echoUser for consumer api-management-test-user", "Adding group mobileAdmin for consumer api-management-test-user", "Adding group mobileApp for consumer api-management-test-user", "Adding group noteAdmin for consumer api-management-test-user", "Adding group noteUser for consumer api-management-test-user", "Adding group orgAdmin for consumer api-management-test-user", "Adding group orgUser for consumer api-management-test-user", "Adding group publicUser for consumer api-management-test-user", "Adding group userAdmin for consumer api-management-test-user", "Adding group configUser for consumer api-management-test-user", "Adding group dataAdmin for consumer api-management-test-user", "Creating jwt credentials for consumer api-management-test-user", "JWT token for api-management-test-user is : <JWT TOKEN>", "Adding consumer player", "Existing groups for consumer player : []", "Required groups for consumer player : [u'appAdmin', u'contentAdmin', u'contentUser', u'courseAdmin', u'courseUser', u'echoUser', u'mobileAdmin', u'mobileApp', u'noteAdmin', u'noteUser', u'orgAdmin', u'orgUser', u'publicUser', u'userAdmin', u'configUser', u'dataAdmin']", "Adding group appAdmin for consumer player", "Adding group contentAdmin for consumer player", "Adding group contentUser for consumer player", "Adding group courseAdmin for consumer player", "Adding group courseUser for consumer player", "Adding group echoUser for consumer player", "Adding group mobileAdmin for consumer player", "Adding group mobileApp for consumer player", "Adding group noteAdmin for consumer player", "Adding group noteUser for consumer player", "Adding group orgAdmin for consumer player", "Adding group orgUser for consumer player", "Adding group publicUser for consumer player", "Adding group userAdmin for consumer player", "Adding group configUser for consumer player", "Adding group dataAdmin for consumer player", "Creating jwt credentials for consumer player", "JWT token for player is : <JWT TOKEN>", "Adding rate_limit for consumer player for API createContent", "rate_limit_plugin_data: {u'config.hour': 1000, u'config.limit_by': u'credential', 'name': 'rate-limiting', 'consumer_id': u'b7b092a2-d0a9-4aa6-b843-0a6b20dc954e'}", "Adding rate_limit for consumer player for API updateContent", "rate_limit_plugin_data: {u'config.hour': 10000, u'config.limit_by': u'credential', 'name': 'rate-limiting', 'consumer_id': u'b7b092a2-d0a9-4aa6-b843-0a6b20dc954e'}", "Adding rate_limit for consumer player for API readContent", "rate_limit_plugin_data: {u'config.hour': 100000, u'config.limit_by': u'credential', 'name': 'rate-limiting', 'consumer_id': u'b7b092a2-d0a9-4aa6-b843-0a6b20dc954e'}", "Adding consumer mobile_admin", "Existing groups for consumer mobile_admin : []", "Required groups for consumer mobile_admin : [u'mobileAdmin']", "Adding group mobileAdmin for consumer mobile_admin", "Creating jwt credentials for consumer mobile_admin", "JWT token for mobile_admin is : <JWT TOKEN>", "Credentials for consumer mobile_admin, key: <MOBILE ADMIN KEY>, secret:<MOBILE ADMIN SECRET>", "Adding consumer mobile_app", "Existing groups for consumer mobile_app : []", "Required groups for consumer mobile_app : [u'mobileApp']", "Adding group mobileApp for consumer mobile_app", "Creating jwt credentials for consumer mobile_app", "JWT token for mobile_app is : <JWT TOKEN>", "Adding consumer mobile_device", "Existing groups for consumer mobile_device : []", "Required groups for consumer mobile_device : [u'contentUser', u'courseUser', u'appAdmin', u'configUser', u'publicUser', u'userAdmin', u'contentAdmin']", "Adding group contentUser for consumer mobile_device", "Adding group courseUser for consumer mobile_device", "Adding group appAdmin for consumer mobile_device", "Adding group configUser for consumer mobile_device", "Adding group publicUser for consumer mobile_device", "Adding group userAdmin for consumer mobile_device", "Adding group contentAdmin for consumer mobile_device", "Creating jwt credentials for consumer mobile_device", "JWT token for mobile_device is : <JWT TOKEN>"]}

PLAY RECAP ************************************************************************************************************************************
localhost                  : ok=8    changed=3    unreachable=0    failed=0

Please let me know if this is alright. Can I go ahead with this ?

@Abhishek-Govula That is good. You can ignore the warnings. Please go ahead with next steps

@endeepak Thank you for you constant help. I was able to deploy the apis successfully. Right now working on setting up the keycloak

I am facing an error after I restart my machine starting the api-manager_kong. The following is the error
# docker service ps api-manager_kong --no-trunc
ID                          NAME                     IMAGE                                                                                NODE                DESIRED STATE       CURRENT STATE                  ERROR                                                               PORTS
ocwmrdldn5opdnzf5ilk0xuv3   api-manager_kong.1       sunbird/kong:0.9.9@sha256:ab348bf41fcd62ec3e31709f04072982546a7151c0a025cc87e906965b69e98b   kmadmin             Ready               Ready less than a second ago
z588t7xc3wbmidgrcpz7gune3    \_ api-manager_kong.1   sunbird/kong:0.9.9@sha256:ab348bf41fcd62ec3e31709f04072982546a7151c0a025cc87e906965b69e98b   kmadmin             Shutdown            Failed 51 seconds ago          "starting container failed: No such network: application_default"
zxbxiohiq05pobiniyoj59vst    \_ api-manager_kong.1   sunbird/kong:0.9.9@sha256:ab348bf41fcd62ec3e31709f04072982546a7151c0a025cc87e906965b69e98b   kmadmin             Shutdown            Failed 18 hours ago            "starting container failed: No such network: application_default"
zyv9zztojc0uq5xb1bkingoqc    \_ api-manager_kong.1   sunbird/kong:0.9.9@sha256:ab348bf41fcd62ec3e31709f04072982546a7151c0a025cc87e906965b69e98b   kmadmin             Shutdown            Failed 18 hours ago            "starting container failed: No such network: application_default"
zxqnb2ab51yh7eq1ymx2ev7qe    \_ api-manager_kong.1   sunbird/kong:0.9.9@sha256:ab348bf41fcd62ec3e31709f04072982546a7151c0a025cc87e906965b69e98b   kmadmin             Shutdown            Failed 36 hours ago            "starting container failed: No such network: application_default"

Can you please help

In your previous comment https://github.com/project-sunbird/sunbird-commons/issues/111#issuecomment-343823981 the application_default network was visible
Please check output of docker network ls now.

@endeepak Thanks for pointing out the problem.
I see this problem on my side because, after system restart, my default network is being connected to a different connection and ip other than the one mentioned in the inventory config. But after I connect back to the network and ip mentioned in the inventory config and restart the service, it is not helping.
I guess restarting the service will not work since the docker swarm has already picked up the application_default.
Is there a way I can reset this application_default after I reconnect to the network ? Or can I restart the swarm itself ?

IP change shouldn't affect networks in docker swarm. You can run the deploy-* scripts again, if you have updated any variable in inventories. Please run deploy-apis and see if it helps.
It it doesn't help, please provide the current error message and output of docker network ls for further debugging

The error I mentioned is after I run deploy-apis.sh i.e
# docker service ps api-manager_kong --no-trunc
ID                          NAME                     IMAGE                                                                                NODE                DESIRED STATE       CURRENT STATE                  ERROR                                                               PORTS
ocwmrdldn5opdnzf5ilk0xuv3   api-manager_kong.1       sunbird/kong:0.9.9@sha256:ab348bf41fcd62ec3e31709f04072982546a7151c0a025cc87e906965b69e98b   kmadmin             Ready               Ready less than a second ago
z588t7xc3wbmidgrcpz7gune3    \_ api-manager_kong.1   sunbird/kong:0.9.9@sha256:ab348bf41fcd62ec3e31709f04072982546a7151c0a025cc87e906965b69e98b   kmadmin             Shutdown            Failed 51 seconds ago          "starting container failed: No such network: application_default"
zxbxiohiq05pobiniyoj59vst    \_ api-manager_kong.1   sunbird/kong:0.9.9@sha256:ab348bf41fcd62ec3e31709f04072982546a7151c0a025cc87e906965b69e98b   kmadmin             Shutdown            Failed 18 hours ago            "starting container failed: No such network: application_default"
zyv9zztojc0uq5xb1bkingoqc    \_ api-manager_kong.1   sunbird/kong:0.9.9@sha256:ab348bf41fcd62ec3e31709f04072982546a7151c0a025cc87e906965b69e98b   kmadmin             Shutdown            Failed 18 hours ago            "starting container failed: No such network: application_default"
zxqnb2ab51yh7eq1ymx2ev7qe    \_ api-manager_kong.1   sunbird/kong:0.9.9@sha256:ab348bf41fcd62ec3e31709f04072982546a7151c0a025cc87e906965b69e98b   kmadmin             Shutdown            Failed 36 hours ago            "starting container failed: No such network: application_default"

Please provide output of ``docker network ls`

# docker network ls
NETWORK ID          NAME                  DRIVER              SCOPE
o1r77vxhymtd        api-manager_default   overlay             swarm
btww7a3r2im8        application_default   overlay             swarm
e7c7ffce0c15        bridge                bridge              local
28bff6631190        docker_gwbridge       bridge              local
718fc03f95b9        host                  host                local
9lvcf0i23w53        ingress               overlay             swarm
ncks03iw1wrb        jenkins_default       overlay             swarm
hqa8zofem4vm        logger                overlay             swarm
17gb2b06kgz5        monitoring            overlay             swarm
852c0fac25c7        none                  null                local
vf7fmsr0hhsk        proxy_default         overlay             swarm

@Abhishek-Govula Could you please run ./deploy/system-check.sh, mask out any private info and share the output? You can run it on the swarm master.
Can you please explain your setup a bit? Is it a single node swarm?

@Abhishek-Govula still encountering problems? or is this resolved?


Shader is black

http://i.imgur.com/cIajww4.png
everything is setup properly, using u5.4.f3 shader looks like this
oh and when i open the project it says it was made in 5.3.4 you said in thread you made it work with 5.4 which is incorrect

It was made in 5.3.4 but that doesn't mean it doesn't work in 5.4. The example scene works for me (and others) in 5.4. Can you try the example scene? Do you have any errors in console or on the AtmosphericScattering component? Also, are we talking about editor or standalone player?

Same issue. To make it works in standalone player i have to preload the shaders.  In the editor the shader doens't works,; black sky.  But it works when i launch the scene in the editor.

I doesn't work in the editor when the game isn't running. Never did. That's a feature :) I'll see if this could be improved when I have time.


warn missing cite for model rdf

I agree, I think we should throw a warning if cite is missing.

Originally posted by @constantinpape in https://github.com/bioimage-io/spec-bioimage-io/issues/470#issuecomment-1196837305
I forgot about the warning. this PR adds it. Only for the model RDF. We could alternatively also add it for all RDFs.

Only for the model RDF. We could alternatively also add it for all RDFs.

I think only for the model RDF is fine for now.


Update: SSHFS-Win.SSHFS-Win version 3.5.20357

[X] Have you signed the Contributor License Agreement?
[X] Have you checked that there aren't other open pull requests for the same manifest update/change?

[ ] Have you tested your manifest locally with winget install --manifest <path>?
[X] Does your manifest conform to the 1.0 schema?

Microsoft Reviewers: Open in CodeFlow

This is a CLI app, so it fails when run with no arguments.

@wingetbot waivers Add Validation-Executable-Error

/azp run


typeof NULL is "object"

The typeof check in the compute function is wrong, because null is also an object in javascript.
So the check should be the following:
if (typeof value !== 'object' || value === null)

there is an if(obj) check before applying the compute function so if obj === null then the if(typeof value !== 'object' statement will be unreachable.


Incorrect CLI usage flags for forest and forest-cli

Describe the bug

After the binaries split was introduced in https://github.com/ChainSafe/forest/issues/1901, some of the flags don't make any sense for some of the binaries, e.g. --import-snapshot for forest-cli or --token for forest (unless I'm missing something)
To Reproduce
Steps to reproduce the behavior:

Run forest-cli
See usage flags.

Log output

Log Output
❯ target/debug/forest-cli --help
forest-cli 0.4.1+git.110e83ad
ChainSafe Systems<EMAIL_ADDRESS>Forest binary sub-commands available

USAGE:
  forest-cli [FLAGS] [OPTIONS] <SUBCOMMAND>

FLAGS:
      --detach               Daemonize Forest process
      --halt-after-import    Halt with exit code 0 after successfully importing a snapshot
  -h, --help                 Prints help information
      --skip-load            Skips loading CAR file and uses header to index chain. Assumes a pre-loaded database
  -V, --version              Prints version information

OPTIONS:
      --chain <chain>
          Choose network chain to sync to [default: mainnet]  [possible values: mainnet, calibnet]

  -c, --config <config>                            A TOML file containing relevant configurations
      --encrypt-keystore <encrypt-keystore>        Encrypt the key-store (default: true)
  -g, --genesis <genesis>                          The genesis CAR file
      --height <height>                            Validate snapshot at given EPOCH
      --import-chain <import-chain>                Import a chain from a local CAR file or URL
      --import-snapshot <import-snapshot>          Import a snapshot from a local CAR file or URL
  -k, --kademlia <kademlia>                        Allow Kademlia (default: true)
      --mdns <mdns>                                Allow MDNS (default: false)
      --metrics-address <metrics-address>
          Address used for metrics collection server. By defaults binds on localhost on port 6116

      --req-window <req-window>                    Number of tipsets requested over chain exchange (default is 200)
  -r, --rpc <rpc>                                  Allow RPC to be active or not (default: true)
      --rpc-address <rpc-address>                  Address used for RPC. By defaults binds on localhost on port 1234
      --target-peer-count <target-peer-count>      Amount of Peers we want to be connected to (default is 75)
      --tipset-sample-size <tipset-sample-size>
          Number of tipsets to include in the sample that determines what the network head is

  -t, --token <token>                              Client JWT token to use for JSON-RPC authentication

SUBCOMMANDS:
  auth            Manage RPC permissions
  chain           Interact with Filecoin blockchain
  config          Manage node configuration
  fetch-params    Download parameters for generating and verifying proofs for given size
  genesis         Work with blockchain genesis
  help            Prints this message or the help of the given subcommand(s)
  mpool           Interact with the message pool
  net             Manage P2P network
  send            Send funds between accounts
  snapshot        Manage snapshots
  state           Interact with and query Filecoin chain state
  sync            Inspect or interact with the chain synchronizer
  wallet          Manage wallet

Expected behaviour

Flags available for forest and forest-cli are only those that make sense and are usable. There should not be any flags that belong solely to another binary.
Screenshots

Environment (please complete the following information):

OS: Fedora 36
Rust version(e.g. rustc --version) rustc 1.66.0-nightly (470e518c4 2022-09-27)
Branch/commit 110e83ad (main)

Other information and links

The options are kept mostly because the cli and daemon share the logic of config file, as per @lemmih the config file is only needed for the daemon, a refactor to remove config logic from cli would help fix this

@jdjaustin Please note one caveat - currently we have a single command to dump the default TOML config - forest-cli config dump. This mostly dumps the daemon configuration (a bit counterintuitive). A slightly more ergonomic approach would be to move it to forest config dump and optionally forest-cli config dump if we want to support config files for the CLI binary.

@LesnyRumcajs do you think we need to keep this issue open? Seems like it's a bit out of date.

It is still relevant. forest-cli exposes a bunch of flags it doesn't really use. On the other hand, we can assume it's superseded by https://github.com/ChainSafe/forest/issues/3172. @lemmih ?

Hmm, let's keep this issue open and use it as a minder to check the flags after #3172 is closed.

I have some ideas about this, I can build a PoC for the team

@LesnyRumcajs The flags look good to me now.
USAGE:
  forest-cli [OPTIONS] <COMMAND>

SUBCOMMANDS:
  chain     Interact with Filecoin blockchain
  auth      Manage RPC permissions
  net       Manage P2P network
  sync      Inspect or interact with the chain synchronizer
  mpool     Interact with the message pool
  state     Interact with and query Filecoin chain state
  config    Manage node configuration
  snapshot  Manage snapshots
  send      Send funds between accounts
  info      Print node info
  attach    Attach to daemon via a JavaScript console
  shutdown  Shutdown Forest
  help      Print this message or the help of the given subcommand(s)

OPTIONS:
  -t, --token <TOKEN>  Client JWT token to use for JSON-RPC authentication
  -h, --help           Print help
  -V, --version        Print version

Could you confirm and close this issue if everything is okay?

There's a small thing I'll resolve in #3958, otherwise LGTM.


oauthlib support

Is there any plan to implement support for oauthlib?

@fafhrd91 I might work on this.  Does aiohttp have some sort of auth extension interface like this?

@dmwyatt it's quite easy to implement this by yourself 1 and 2. What the underlying library or auth method will be used is up to your requirements.

@dmwyatt aiohttp does not have support for custom auth. but PR are welcome!

I've written aioauth-client library https://github.com/klen/aioauth-client. It in beta version now, but should be tested. For my cases (github and twitter authorization, api requests) it works well.

thank you @klen, I'll test that in the future because we need to implement an integration with Office365.


Add AtlantaFX to the libraries

Thanks for the PR. May I ask you for a change please? JPro is not a big fan of animated GIFs ... it simply consumes too much memory. Could you turn this into a standard image and then maybe add several of them? Also: a size of 800x600 is usually enough for such previews.

Sorry, didn't know about that. I submitted new PR.


Automate Event Generation

Could you please provide more information on how you can automate Event Generation using REST APIs over HTTPS for companies which are using MFA (as Basic Auth will not work). Also, provide permissions required to generate events.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 1ad33cc1-dc5f-d9f9-ee8e-b103839945ef
Version Independent ID: 1bc3101d-afe8-2056-3cda-6a86a209c460
Content: Start retention when an event occurs - Microsoft 365 Compliance
Content Source: microsoft-365/compliance/event-driven-retention.md
Service: o365-seccomp
GitHub Login: @cabailey
Microsoft Alias: cabailey

After checking with engineering, I'm told we support basic auth only at the moment but looking at integrating with MS Graph in the future. It would be very helpful to raise this request by submitting it to UserVoice - this category: https://office365.uservoice.com/forums/289138-office-365-security-compliance?category_id=379531
The permissions required to use the REST APIs are the same as manually creating an event using the UI - for example, you can use Compliance Administrator.