Skip to content
GitLab
Menu
Projects
Groups
Snippets
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
John-Paul Robinson
createAndParseSACCT
Commits
d519057b
Commit
d519057b
authored
Mar 31, 2020
by
William Stonewall Monroe
Browse files
stripping notebook info
parent
cf3a83da
Changes
2
Hide whitespace changes
Inline
Side-by-side
importSACCTinfo.ipynb
View file @
d519057b
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
python
```
import numpy as np
import numpy as np
import pandas as pd
import pandas as pd
import pandas_profiling
import pandas_profiling
```
```
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
python
```
df = pd.read_csv('userusage.txt',delimiter='|')
df = pd.read_csv('userusage.txt',delimiter='|')
```
```
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
python
```
df.head()
df.head()
```
```
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
python
```
df[['jid','step']] = df.JobID.str.split(".",expand=True)
df[['jid','step']] = df.JobID.str.split(".",expand=True)
df.Partition.values
df.Partition.values
```
```
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
python
```
batchDF=df.dropna(subset=["MaxRSS"])
batchDF=df.dropna(subset=["MaxRSS"])
userDF=df.dropna(subset=["User"])
userDF=df.dropna(subset=["User"])
for jid in df.jid.unique():
for jid in df.jid.unique():
userDF['MaxRSS'][userDF['jid'] == jid]=batchDF['MaxRSS'][batchDF['jid'] == jid]
userDF['MaxRSS'][userDF['jid'] == jid]=batchDF['MaxRSS'][batchDF['jid'] == jid]
#print(userDF[userDF['jid'] == jid])
#print(userDF[userDF['jid'] == jid])
userDF.head()
userDF.head()
```
```
%% Cell type:markdown id: tags:
%% Cell type:markdown id: tags:
# add more graphs here
# add more graphs here
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
python
```
```
```
...
...
slurm-2sql.ipynb
View file @
d519057b
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
python
```
import sqlite3
import sqlite3
import slurm2sql
import slurm2sql
import pandas as pd
import pandas as pd
```
```
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
python
```
db = sqlite3.connect('test.db')
db = sqlite3.connect('test.db')
slurm2sql.slurm2sql(db, ['-S', '2020-03-18', '-a'])
slurm2sql.slurm2sql(db, ['-S', '2020-03-18', '-a'])
```
```
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
python
```
# For example, you can then convert to a dataframe:
# For example, you can then convert to a dataframe:
df1 = pd.read_sql('SELECT * FROM slurm', db)
df1 = pd.read_sql('SELECT * FROM slurm', db)
```
```
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
python
```
df1.head(5)
df1.head(5)
```
```
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
python
```
print("more plots to come")
print("more plots to come")
```
```
%% Cell type:code id: tags:
%% Cell type:code id: tags:
```
python
```
```
```
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment