TL;DR - I can’t get AnsibleContainer to build a FutureGateway stack. Maybe documenting my descent into insanity will help me get out of it.

Every now and again, has the feeling of just being stuck. There seems to be no trodden path leading to the point in which one finds oneself, no direction in or out - one has the distinct impression of waking up from something like a dream and realising that one has no idea how one arrived.

“Was I drunk ? “, “Was there some kind of party ? “, “Who won ? “… although futile, one chases the tumbleweed of thoughts in the hope of finding out where to make the next step.

Well, at that point, any step is the right step ! So, here goes 1 …

Get me out of here

I’ve been working on this damn problem for over a year now, and it seems utterly ridiculous that I can’t bring it to a conclusion. “What problem ?”, I hear you ask (or is that the hallucinations talking ?).

1

  1. Started this blog post
  2. Deleted all dist packages of ansible-container
  3. Deleted all docker images and containers
  4. Re-installed ansible-container with pip (now gives 0.9.1)
  5. Ran ansible-container build - got the errors below (root not in passwd file)
  6. Fixed that by pulling down the image first.

Next ? re-run the build part and identify the next error.

2

  1. Ensure that we are using a debian-based conductor
  2. Run the build again (encounter “unreachable/authentication” problem below)
  3. Found out where to include the MySQL-python package to allow the mysql_db module - /usr/local/lib/python2.7/dist-packages/container/docker/templates/conductor-dockerfile.j2 , so that’s good !

3

  1. Change user in the db role
  2. Check out ansible-container from git again.
  3. When I change the base image of the db service from mysql to the same as the conductor image, this problem disappears
  4. ok, playbook seems to work ok, but only using CEntOS as base image

Oh yeah that makes perfect sense… Except no it frikkin doesn’t ! What’s the point of having to have a homogenous cluster of base images ? I’m getting too old for this

4

  1. Try to make the mysql server on CEntOS Terrible idea - way too much work already done by others.
  2. Found centos/mysql-56-centos7 - let’s try that
  3. Use it on the command line with docker run -d -e MYSQL_ROOT_PASSWORD=password -e MYSQL_USER=user -e MYSQL_PASSWORD=pass -e MYSQL_DATABASE=fgapiserver centos/mysql-57-centos7 - that seems to create the db.
  4. Nope, that doesn’t seem to get the db started : “msg”: “unable to find /var/lib/mysql/.my.cnf. Exception message: (2002, "Can’t connect to local MySQL server through socket ‘/var/lib/mysql/mysql.sock’ (2)")”
  5. Aaaand, nowhere is the frikkin Dockerfile documented for this image

5

  1. Ok, wait docker inspect has my back 2 - but adding the entrypoint doesn’t actually give me a working db to access.
  2. At this point… I’m pretty much losing hope that AnsibleContainer can do this. Let’s go read the example again.
  3. Ok, wordpress example (which is pre-0.9.1 however) says basically “start mysqld_safe in the background and tune db according to normal Ansible procedure.”

Maybe it’s time to go back to the “Ansible playbook against docker endpoint” strategy. Sigh, it feels like a defeat at this point…

What am I even building ?

Look, call me unreasonable, but it seems reasonable to me to be able to define a topology and configuration a-la 12 factor , and then have AnsibleContainer whip me up the images. The idea of “take this base application image (mysql, apache, java) and apply the following configuration to them respectively” just doesn’t seem to be really that practical. I’ve spent the last 3 days hacking away at this, and I’m no closer than when I started.

So I’m going to switch tack and try the old-school way.

Notes

  • I kept getting the error on building the db container :
                  
                    Building
                    service
                    ...
                    project
                    =
                    AnsibleContainer
                    service
                    =
                    db
                    Traceback
                    (
                    most
                    recent
                    call
                    last
                    ):
                    File
                    "/usr/bin/conductor"
                    ,
                    line
                    11
                    ,
                    in
                    <
                    module
                    >
                    load_entry_point
                    (
                    'ansible-container'
                    ,
                    'console_scripts'
                    ,
                    'conductor'
                    )()
                    File
                    "/_ansible/container/__init__.py"
                    ,
                    line
                    19
                    ,
                    in
                    __wrapped__
                    return
                    fn
                    (
                    *
                    args
                    ,
                    **
                    kwargs
                    )
                    File
                    "/_ansible/container/cli.py"
                    ,
                    line
                    360
                    ,
                    in
                    conductor_commandline
                    **
                    params
                    )
                    File
                    "/_ansible/container/__init__.py"
                    ,
                    line
                    19
                    ,
                    in
                    __wrapped__
                    return
                    fn
                    (
                    *
                    args
                    ,
                    **
                    kwargs
                    )
                    File
                    "/_ansible/container/core.py"
                    ,
                    line
                    711
                    ,
                    in
                    conductorcmd_build
                    container_id
                    =
                    engine
                    .
                    run_container
                    (
                    cur_image_id
                    ,
                    service_name
                    ,
                    **
                    run_kwargs
                    )
                    File
                    "/_ansible/container/docker/engine.py"
                    ,
                    line
                    82
                    ,
                    in
                    __wrapped__
                    return
                    fn
                    (
                    self
                    ,
                    *
                    args
                    ,
                    **
                    kwargs
                    )
                    File
                    "/_ansible/container/__init__.py"
                    ,
                    line
                    19
                    ,
                    in
                    __wrapped__
                    return
                    fn
                    (
                    *
                    args
                    ,
                    **
                    kwargs
                    )
                    File
                    "/_ansible/container/docker/engine.py"
                    ,
                    line
                    196
                    ,
                    in
                    run_container
                    **
                    run_kwargs
                    File
                    "/usr/lib/python2.7/site-packages/docker/models/containers.py"
                    ,
                    line
                    700
                    ,
                    in
                    run
                    container
                    .
                    start
                    ()
                    File
                    "/usr/lib/python2.7/site-packages/docker/models/containers.py"
                    ,
                    line
                    333
                    ,
                    in
                    start
                    return
                    self
                    .
                    client
                    .
                    api
                    .
                    start
                    (
                    self
                    .
                    id
                    ,
                    **
                    kwargs
                    )
                    File
                    "/usr/lib/python2.7/site-packages/docker/utils/decorators.py"
                    ,
                    line
                    21
                    ,
                    in
                    wrapped
                    return
                    f
                    (
                    self
                    ,
                    resource_id
                    ,
                    *
                    args
                    ,
                    **
                    kwargs
                    )
                    File
                    "/usr/lib/python2.7/site-packages/docker/api/container.py"
                    ,
                    line
                    1070
                    ,
                    in
                    start
                    self
                    .
                    _raise_for_status
                    (
                    res
                    )
                    File
                    "/usr/lib/python2.7/site-packages/docker/api/client.py"
                    ,
                    line
                    216
                    ,
                    in
                    _raise_for_status
                    raise
                    create_api_error_from_http_exception
                    (
                    e
                    )
                    File
                    "/usr/lib/python2.7/site-packages/docker/errors.py"
                    ,
                    line
                    31
                    ,
                    in
                    create_api_error_from_http_exception
                    raise
                    cls
                    (
                    e
                    ,
                    response
                    =
                    response
                    ,
                    explanation
                    =
                    explanation
                    )
                    docker
                    .
                    errors
                    .
                    APIError
                    :
                    500
                    Server
                    Error
                    :
                    Internal
                    Server
                    Error
                    (
                    "linux spec user: unable to find user root: no matching entries in passwd file"
                    )
                    Conductor
                    terminated
                    .
                    Cleaning
                    up
                    .
                    command_rc
                    =
                    1
                    conductor_id
                    =
                    06061592e5
                    d81258ce2e11a29d9b251b9f4fb5040955ff0bff1b80f06b856472
                    save_container
                    =
                    False
                  
                
  • Now, I get an error with authentication issues ?
                  
                    {
                    "changed"
                    : 
                    false
                    ,          
                    "msg"
                    : 
                    "Authentication or permission failure. In some cases, you may have been able to authenticate and did not have permissions on the remote directory. Consider changing the remote temp path in ansible.cfg to a path rooted in 
                    \"
                    /tmp
                    \"
                    . Failed command was: ( umask 77 && mkdir -p 
                    \"
                    `
                    echo
                     ~/.ansible/tmp/ansible-tmp-1496051043.32-155874901509998 
                    `
                    \"
                     && echo ansible-tmp-1496051043.32-155874901509998=
                    \"
                    `
                    echo
                     ~/.ansible/tmp/ansible-tmp-1496051043.32-155874901509998 
                    `
                    \"
                     ), exited with result 1"
                    ,          
                    "unreachable"
                    : 
                    true
                    }
                  
                

Summary of issues encountered

Description Error Diagnosis
root user not in image docker.errors.APIError: 500 Server Error: Internal Server Error ("linux spec user: unable to find user root: no matching entries in passwd file") base image was missing and didn’t get pulled down
Authentication or permission error Authentication or permission failure. In some cases, you may have been able to authenticate and did not have permissions on the remote directory. Consider changing the remote temp path in ansible.cfg to a path rooted in \"/tmp\". Seems to have been fixed in [#236](https://github.com/ansible/ansible-container/pull/236/commits/aee073b9763097b1b57cf425ec94314746046456) , but not in 0.9.1 Seems not present in CEntOS base images (Same as conductor)

Footnotes and References

                  
                    
                    "Cmd"
                    :
                    
                    [
                    
                    "run-mysqld"
                    
                    ]
                    ,
                    
                    "ArgsEscaped"
                    :
                    
                    true
                    ,
                    
                    "Image"
                    :
                    
                    "sha256:fd6ae9ba11bb2c2f7d4196b6c3f7e786a7b1343bb76fd6d8a132ba207c66901a"
                    ,
                    
                    "Volumes"
                    :
                    
                    {
                    
                    "/var/lib/mysql/data"
                    :
                    
                    {}
                    
                    }
                    ,
                    
                    "WorkingDir"
                    :
                    
                    ""
                    ,
                    
                    "Entrypoint"
                    :
                    
                    [
                    
                    "container-entrypoint"
                    
                    ]
                    
                  
                
  1. Note to the reader - this is purely an attempt to keep some semblance of writing skills. It’s probably going to be quite painful to read, since it’s not intended for you.  ↩

  2. HT this docker forum thread :  ↩