README.md 8.32 KB
Newer Older
Mitchell Moore's avatar
Mitchell Moore committed
1
2
3
# Introduction
This is a flask application that leverages [RabbitMQ](https://www.rabbitmq.com/) and [Celery](https://docs.celeryproject.org/en/latest/getting-started/introduction.html) 
to asynchronously create a Cheaha user account. Currently the project is being developed on an Openstack cluster. 
4
5
6
7
8
# Project Setup 


To clone this repo use the command: 
```
9
$ git clone https://gitlab.rc.uab.edu/mmoo97/flask_user_reg.git
Mitchell Moore's avatar
Mitchell Moore committed
10
$ cd flask_user_reg
11
12
```
## Prerequisites 
Mitchell Moore's avatar
Mitchell Moore committed
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
### Setup a Virtual Environment
- Ensure you have created a [virtual environment](https://docs.python.org/3/library/venv.html) 
called `venv` setup running python3.
  - Note, this project requires a virtual environment running __python3__ (__3.6.8__ in this case).
    - Create this by navigating to you home directory via typing `$ cd` and entering the following commands:<br>
    ```
    $ python3 -m venv ~/venv
    $ source ~/venv
    ``` 
    - Upon Activation, you should see the prompt update accordingly:
    ```
    [centos@ood ~]$              <------Old Prompt
    (venv) [centos@ood ~]$       <------New Prompt
    ```
    In this case, the env name is displayed as `venv` but would change to reflect whatever name you initialized it with
    in the previous step. Additionally, this example is running on the `ood node` provisioned
    via __OpenStack__.
- Ensure [pip](https://docs.python.org/3/installing/index.html) is installed.
  - #### Mac/Linux
    - Check if installed by typing `$ pip`
    - Install pip using `$ python -m pip install --user --upgrade pip`.
  - #### Windows
    - Check if installed using `$ py`
    - Install pip using `$ py -m pip install --upgrade pip`<br><br>
- Ensure Flask and other dependencies are installed to you virtual environment  using the following commands:
38
```
39
40
$ cd ~/your/repo/path/flask_user_reg
$ pip install -r requirements.txt
41
```
Mitchell Moore's avatar
Mitchell Moore committed
42
-  Note, to install flask in your own `$HOME` use `pip install --user Flask`.
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
### Install RabbitMQ
(Reference: [here](https://www.rabbitmq.com/tutorials/tutorial-four-python.html))
- Install RabbitMQ server on the host machine. ([Installation Guide](https://www.rabbitmq.com/download.html))
  - #### Mac/Linux
    - `$ brew update`
    - `$ brew install rabbitmq`
    - it is recommended that you add the following line to your `.bash_profile`:  
 `export PATH=$PATH:/usr/local/opt/rabbitmq/sbin`.
    - Start server using the command `$ rabbitmq-server`. (Note, this implementation assumes RabbitMQ is running on localhost on standard port 5672)
  - #### Windows
    - Download the installer from [here](https://github.com/rabbitmq/rabbitmq-server/releases) and run.
    - Post install, the server should be running. To check, run `$ rabbitmqctl.bat status`.
  - #### CentOS 7
    - First, import signing key using `$ rpm --import https://github.com/rabbitmq/signing-keys/releases/download/2.0/rabbitmq-release-signing-key.asc`
    - Next, install Erlang using `$ yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm`
    - Finally, install RabbitMq using `$ yum install rabbitmq-server-3.8.2-1.el8.noarch.rpm`
    - The server is not started as a daemon by default when the RabbitMQ server package is installed. To start the 
    daemon by default when the system boots, as an administrator run: `$ chkconfig rabbitmq-server on`.
    - As an administrator, start and stop the server as usual: <br/>`/sbin/service rabbitmq-server start` 
    <br/> `/sbin/service rabbitmq-server stop`
 - Add the user "reggie" using the command `$ rabbitmqctl add_user reggie reggie`.
   - The current configuration assumes the following username password combination. To change the password, type 
   `$ rabbitmqctl change_password reggie <new_password>`. 
   - Note that rabbitmqctl may require sudo permissions and that changing the password will require a password
   change in the credentials within `test_producer.py` and `base_consumer.py` as well.
Mitchell Moore's avatar
Mitchell Moore committed
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
# Run Project

## 1. Connect to OHPC/OOD
1. Network Setup: <br>
    Assuming proper steps have been run to provision your openstack cluster, your network
    topology should look like the following:<br>
    ![Openstack Network Topology](/docs/images/network_topology.png)

2.  Locate Floating Ip for each instance: <br>
    ![Research Computing Network](/docs/images/rc-network%20.png)<br>
    According to this diagram of the network infrastructure, the IP Addresses that you will need to 
    connect to in order to ssh into each machine will be `164.111.161.xxx` where
    `.xxx` is the last octet of your assigned Floating IP Addresses shown below.<br>
    ![Floating IP Addresses](/docs/images/floating_ips.png)<br>
    Note that these numbers will typically vary depending on factors in the provisioning 
    process.
84

Mitchell Moore's avatar
Mitchell Moore committed
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
3. SSH into each machine: <br>
    Now that you have the IP Addresses of the OHPC and OOD nodes, you can connect
    to them via the following command in two separate shell windows/tabs: <br>
    ```
   $ ssh centos@164.111.161.xxx
    ```

4. Drop Firewall: <br>
 Currently, the firewall prevents us from connecting to our soon to be running flask app. 
 To check if the firewall is active, type: <br>
 `$ sudo systemctl status firewalld` <br> 
 In the case the firewall is still active, type: <br>
 `$ sudo systemctl stop firewalld` <br>
 In the case you need to turn the firewall back on, type: <br> 
 `sudo systemctl start firewalld`

5.  Edit Security Rules: <br> 
 Lastly, we want to make sure that our network it open to run on our flask application
 which runs on `localhost:5000` by default. Modify your __Default__ security group to 
 reflect the following rules: <br> 
 ![Security Rules](/docs/images/security_rules.png) <br>
 The main takeaway/modification in these rules is that traffic is allowed in and out of port 5000.
 

## 2. Initialize RabbitMQ
Make sure that you are running rabbitmq via the command `sudo rabbitmq server`.
## 3. Test RabbitMQ
112
113
114
115
116
117
118
119
120
121
122
123
124
For a simple example on the functionality of RabbitMQ, do the following:
- Open up a new ssh terminal and ssh into your ohpc node and, in a separate window, ssh into your ood node.
- Additionally, ensure you have a rabbitmq user configured with the username and password as `reggie`.

```
# Run consumer on ohpc node
$ python base_consumer.py ohpc
```
 You will notice that the script is awaiting a message to display.
 To produce a message, run the following on the ood node terminal session:
 ```
$ python test_producer.py ohpc 
```
125

126
You should now see that the message has been sent and displayed on the ohpc node.
Mitchell Moore's avatar
Mitchell Moore committed
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161

## 4.  Celery Worker
In order to execute our tasks asychronously, we create a celery worker using the fllowing: <br>
`$ celery -A tasks worker --loglevel=info --concurrency=4 1> ~/celery.out 2> ~/celery.err &` <br><br>
In this case, the `celery -A tasks worker --loglevel=info --concurrency=4` portion of the command
is what is initiating the worker and the remainder serves to write the stdout and stderr to two separate files 
located in the home directory and to run the process in the background. <br>


## 5. Initialize the Flask App
Simply type `$ python run.py` <br>
Alternatively, type `$ python run.py > ~/flask.out 2> ~/flask.err &` to run the process in the background
and output to files in the home directory.

## 6. Check Job status    

If all goes well, you should have your processes running in the background. Check the status of these by typing 
`$ jobs`. You should now see the something similar to the following: <br> 
```
(venv) [centos@ood flask_user_reg]$ jobs
[1]-  Running                 celery -A tasks worker --loglevel=info --concurrency=4 > ~/celery.out 2> ~/celery.err &
[2]+  Running                 python run.py > ~/server.out 2> ~/server.err &
(venv) [centos@ood flask_user_reg]$ 
```

## 7. Connect to Server
Open a new browser window and connect to the __OOD__ node by typing `http://164.111.161.xxx:5000` in the
address bar replacing `.xxx` with the final octet of the __OOD__ node. You should see something like this: <br>
![Self Registration Site](/docs/images/site_example.png) <br>
## 8. Key Info/Observe
- Fill out the form and hit "Submit"
- You should see an overlay signifying the account is being created. After 5 seconds it should disappear and signify the account creation has been successful.
- You can now enter the command `kill %1 %2` to terminate the celery worker and the flask server.
- You can view the output/errors of either the worker and flask server by opening `flask.out`, `flask.err`, 
`celery.out`, or `celery.err`.