Cloud Foundry BOSH troubleshoot – cannot access director

I was upgrading Pivotal Cloud Foundry from 1.4 to 1.5.
After importing the Ops Manager 1.5 ova and then importing the previous installation, I clicked Apply Changes to kick of the upgrade. However, in the middle of installation, the disk ran out of space. So I deleted the imported Ops Manager 1.5 (which uses Thick Provisioning Lazy-Zeroed) from vSphere, and re-imported it with Thin Provisioned storage to save space.

I repeated the normal steps to upgrade PCF, I ran into issue with the missing VM.
Error:

/home/tempest-web/tempest/web/vendor/bundle/ruby/2.2.0/gems/bosh_vsphere_cpi-1.3026.0/lib/cloud/vsphere/vm_provider.rb:11:in `find': <strong>VM `vm-d23556f9-4081-4c03-978b-14442d399525' not found (Bosh::Clouds::VMNotFound)</strong>

Well, the problem is that it exited in the middle of an installation. The old installation zip file, which contains bosh_deployment.yml, refers to a VM that no longer exist.

So next step is to remove that

ubuntu@pivotal-ops-manager:~$ bosh target 172.18.235.21

Error:

[WARNING] cannot access director, trying 4 more times...
[WARNING] cannot access director, trying 3 more times...
[WARNING] cannot access director, trying 2 more times...
[WARNING] cannot access director, trying 1 more times...
cannot access director <strong>(Connection refused - connect(2) for "172.18.235.21" port 25555 </strong>(https://172.18.235.21:25555))

 

On Micro bosh VM

netstat –listen

Problem: the bosh VM is not listening on port 25555

 

monit stop all

monit start all

netstat –listen | grep 25555

tcp    0    0  :25555    :*    LISTEN

hey it’s back!

now try connecting from Ops Manager again

bosh target YOUR_DIRECTOR_IP

Target set to ‘microbosh-8be57082b0a794b1b750’

Your username: directory

Enter password:

Logged in as ‘director’

YAY.

reference: Restoring Pivotal Cloud Foundry After Disaster