bfh/cas-pml
Archived
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
This repository has been archived on 2026-06-23. You can view files and clone it. You cannot open issues or pull requests or push a commit.
Files
688f40feeb49698127fcafb8e2c6dcc3cf023e6c
cas-pml/SL/aufgaben/template/2_Code/5 Deployment und Abschluss.ipynb
T
aaron 41e15ed275 refactor: move things around
2026-05-21 14:16:30 +02:00

15 KiB
Raw Blame History

Feature Engineering

Klassifikation

Regression

Validierung und mehr

Deployment und Abschluss

In [1]:
import sys
sys.path.append('./')
In [2]:
## prepare environment
import pandas as pd
import numpy as np
datapath = '../3_data'
from os import chdir; chdir(datapath)

Das finale Modell

Feature Engineering in der Produktion

Missing Values

Neue Kategorien

Protokollieren

In [3]:
data = pd.read_csv('bank_data.csv', sep=';')

import datetime

f = open('fe_prod_log.log','w')
f.write(datetime.datetime.now().strftime("[%Y-%m-%d %H:%M:%S] (timestamp)"))

f.write('\n\n')

s = data.isna().sum()
f.write('features with NA\'s\n')
f.write('=======================')

f.write('\n')
f.write(s[s > 0].to_string())
f.write('\n\n')

## value counts of not numeric features
f.write('categorical cols levels\n')
f.write('=======================')
f.write('\n')
catcolnames = data.select_dtypes(include='object').columns
for ccn in catcolnames:
    f.write(ccn)
    f.write('\n')
    f.write(data[ccn].value_counts().sort_index().to_string())
    f.write('\n\n')
f.close()

Modellübergabe in die Prodkution

Modelle speichern scikit-learn intern

In [4]:
## load data
from bfh_cas_pml import prep_data
X_train, X_test, y_train, y_test = prep_data('melb_data_prep.csv', 'Price')
In [5]:
## three models:

## StandardScaler
from sklearn.preprocessing import StandardScaler
model_sc = StandardScaler().fit(X_train)

## LinearRegression
from sklearn.linear_model import LinearRegression
model_lr = LinearRegression().fit(X_train, y_train)

## DecisionTreeRegressor
from sklearn.tree import DecisionTreeRegressor
model_dt = DecisionTreeRegressor(max_depth= 2, random_state=1234).fit(X_train, y_train)
In [6]:
## save models
import pickle 
with open('model_sc.pkl', 'wb') as pickle_file:
    pickle.dump(model_sc, pickle_file)
with open('model_lr.pkl', 'wb') as pickle_file:
    pickle.dump(model_lr, pickle_file)
with open('model_dt.pkl', 'wb') as pickle_file:
    pickle.dump(model_dt, pickle_file)
In [7]:
## reload models
with open('model_sc.pkl', 'rb') as pickle_file:
    model_sc_2 = pickle.load(pickle_file)
with open('model_lr.pkl', 'rb') as pickle_file:
    model_lr_2 = pickle.load(pickle_file)
with open('model_dt.pkl', 'rb') as pickle_file:
    model_dt_2 = pickle.load(pickle_file)
In [8]:
## compare model_sc
print(model_sc.mean_, '\n')
print(model_sc.var_, '\n')
print(model_sc_2.mean_, '\n')
print(model_sc_2.var_, '\n')
[ 2.93174034e+00  1.45261784e+00  1.04267493e+01  1.43475779e+00
  1.68210732e+00  2.36615297e+00  2.09901978e+00  1.96804559e+03
  5.74824662e+00 -3.78091214e+01  1.44997478e+02  7.52039447e+03
  6.57396836e-01  1.29098026e-01  9.18284130e-02  2.89430762e-01
  3.73511662e-02  3.43010928e-01  1.01125428e-02  2.10650791e-01
  7.15274833e+00  2.01656214e+03  4.83925950e+00] 

[9.08813135e-01 4.68436712e-01 3.68004376e+01 4.24996430e-01
 7.32413501e-01 7.11803450e-01 3.34340977e-02 6.45488299e+02
 3.80781143e+01 5.53828210e-03 9.28052881e-03 2.02869238e+07
 2.25226236e-01 1.12431726e-01 8.33959556e-02 2.05660596e-01
 3.59560566e-02 2.25354431e-01 1.00102793e-02 1.66277035e-01
 6.21684084e+00 2.46138222e-01 1.23097215e+00] 

[ 2.93174034e+00  1.45261784e+00  1.04267493e+01  1.43475779e+00
  1.68210732e+00  2.36615297e+00  2.09901978e+00  1.96804559e+03
  5.74824662e+00 -3.78091214e+01  1.44997478e+02  7.52039447e+03
  6.57396836e-01  1.29098026e-01  9.18284130e-02  2.89430762e-01
  3.73511662e-02  3.43010928e-01  1.01125428e-02  2.10650791e-01
  7.15274833e+00  2.01656214e+03  4.83925950e+00] 

[9.08813135e-01 4.68436712e-01 3.68004376e+01 4.24996430e-01
 7.32413501e-01 7.11803450e-01 3.34340977e-02 6.45488299e+02
 3.80781143e+01 5.53828210e-03 9.28052881e-03 2.02869238e+07
 2.25226236e-01 1.12431726e-01 8.33959556e-02 2.05660596e-01
 3.59560566e-02 2.25354431e-01 1.00102793e-02 1.66277035e-01
 6.21684084e+00 2.46138222e-01 1.23097215e+00]
In [9]:
## compare model_lr
print(model_lr.intercept_)
print(model_lr.coef_, '\n')
print(model_lr_2.intercept_)
print(model_lr_2.coef_)
-148600153.0644718
[ 2.38730932e+05 -1.44127046e+05 -4.11587415e+04  1.55233185e+05
  4.16082458e+04  8.22944764e+04  2.97716907e+05 -2.76754904e+03
 -4.92937355e+03 -5.11115932e+05  1.79228159e+05 -1.58740210e+00
  7.07758671e+04  7.99854850e+03  3.38953273e+04 -1.73563804e+05
  1.07806729e+05  2.39261230e+05  2.81825047e+05 -2.26744902e+05
  1.27253408e+03  5.38476338e+04  3.83311293e+03] 

-148600153.0644718
[ 2.38730932e+05 -1.44127046e+05 -4.11587415e+04  1.55233185e+05
  4.16082458e+04  8.22944764e+04  2.97716907e+05 -2.76754904e+03
 -4.92937355e+03 -5.11115932e+05  1.79228159e+05 -1.58740210e+00
  7.07758671e+04  7.99854850e+03  3.38953273e+04 -1.73563804e+05
  1.07806729e+05  2.39261230e+05  2.81825047e+05 -2.26744902e+05
  1.27253408e+03  5.38476338e+04  3.83311293e+03]
In [10]:
## compare model_dt
from sklearn.tree import export_text
print(export_text(
    model_dt, feature_names=list(X_train.columns)))
print(export_text(
    model_dt_2, feature_names=list(X_train.columns)))
|--- Rooms <= 3.50
|   |--- Type <= 1.50
|   |   |--- value: [1077045.50]
|   |--- Type >  1.50
|   |   |--- value: [682863.38]
|--- Rooms >  3.50
|   |--- Regionname_Southern_Metropolitan <= 0.50
|   |   |--- value: [1163850.50]
|   |--- Regionname_Southern_Metropolitan >  0.50
|   |   |--- value: [2113002.90]

|--- Rooms <= 3.50
|   |--- Type <= 1.50
|   |   |--- value: [1077045.50]
|   |--- Type >  1.50
|   |   |--- value: [682863.38]
|--- Rooms >  3.50
|   |--- Regionname_Southern_Metropolitan <= 0.50
|   |   |--- value: [1163850.50]
|   |--- Regionname_Southern_Metropolitan >  0.50
|   |   |--- value: [2113002.90]

Modelle speichern extern mit PMML

Vorbereitungen:

  • Java Runtime muss vorhanden sein
  • Library installieren direkt in Notebook:
    !pip install sklearn2pmml
In [11]:
!pip install sklearn2pmml
Requirement already satisfied: sklearn2pmml in /Users/vgv1/.pyenv/versions/teaching/lib/python3.13/site-packages (0.119.1)
Requirement already satisfied: dill>=0.3.4 in /Users/vgv1/.pyenv/versions/teaching/lib/python3.13/site-packages (from sklearn2pmml) (0.4.0)
Requirement already satisfied: joblib>=0.13.0 in /Users/vgv1/.pyenv/versions/teaching/lib/python3.13/site-packages (from sklearn2pmml) (1.5.1)
Requirement already satisfied: pandas>=1.5.0 in /Users/vgv1/.pyenv/versions/teaching/lib/python3.13/site-packages (from sklearn2pmml) (2.2.3)
Requirement already satisfied: scikit-learn>=1.0 in /Users/vgv1/.pyenv/versions/teaching/lib/python3.13/site-packages (from sklearn2pmml) (1.6.1)
Requirement already satisfied: numpy>=1.26.0 in /Users/vgv1/.pyenv/versions/teaching/lib/python3.13/site-packages (from pandas>=1.5.0->sklearn2pmml) (2.2.6)
Requirement already satisfied: python-dateutil>=2.8.2 in /Users/vgv1/.pyenv/versions/teaching/lib/python3.13/site-packages (from pandas>=1.5.0->sklearn2pmml) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /Users/vgv1/.pyenv/versions/teaching/lib/python3.13/site-packages (from pandas>=1.5.0->sklearn2pmml) (2025.2)
Requirement already satisfied: tzdata>=2022.7 in /Users/vgv1/.pyenv/versions/teaching/lib/python3.13/site-packages (from pandas>=1.5.0->sklearn2pmml) (2025.2)
Requirement already satisfied: six>=1.5 in /Users/vgv1/.pyenv/versions/teaching/lib/python3.13/site-packages (from python-dateutil>=2.8.2->pandas>=1.5.0->sklearn2pmml) (1.17.0)
Requirement already satisfied: scipy>=1.6.0 in /Users/vgv1/.pyenv/versions/teaching/lib/python3.13/site-packages (from scikit-learn>=1.0->sklearn2pmml) (1.15.3)
Requirement already satisfied: threadpoolctl>=3.1.0 in /Users/vgv1/.pyenv/versions/teaching/lib/python3.13/site-packages (from scikit-learn>=1.0->sklearn2pmml) (3.6.0)
In [12]:
## import libraries
from sklearn.pipeline import Pipeline
from sklearn2pmml import PMMLPipeline, sklearn2pmml
In [13]:
## StandardScaler
from sklearn.preprocessing import StandardScaler
pipeline = PMMLPipeline([("scaler", StandardScaler())]).fit(X_train)
sklearn2pmml(pipeline, "StandardScaler_melb.pmml", with_repr = True)
In [14]:
## LinearRegression
pipeline = PMMLPipeline([('regressor', LinearRegression())]).fit(X_train, y_train)
sklearn2pmml(pipeline, "LinearRegression_melb.pmml", with_repr = True)
In [15]:
## DecisionTreeClassifier
pipeline = PMMLPipeline([('regressor', DecisionTreeRegressor(max_depth= 2, random_state=1234))]).fit(X_train, y_train)
sklearn2pmml(pipeline, "DecisionTreeRegressor_melb.pmml", with_repr = True)
Cell:
[Cell type raw - unsupported, skipped]

Der Modellierungsprozess

(kein Code unter diesem Titel)

pyCaret

vgl. 5.5 Deployment und Abschluss - pycaret.ipynb

Reference in New Issue View Git Blame Copy Permalink