Esta presentación ha sido compuesta utilizando el motor B6+ del W3C, con scripts y estilos modificados por David Charte. Consulta el código fuente en GitHub.
Tesis para la obtención del título de doctor por la Universidad de Granada
Programa de Doctorado en Tecnologías de la
Información y la Comunicación
Francisco David Charte Luque <fdavidcl@ugr.es>
Directores:
Francisco Herrera Triguero
Francisco Charte Ojeda
6 de julio de 2022
Charte, D., Charte, F., García, S., del Jesus, M. J., & Herrera, F. (2018). A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines. Information Fusion, 44, 78-96.
Charte, D., Charte, F., del Jesus, M. J., & Herrera, F. (2020). An analysis on the use of autoencoders for representation learning: Fundamentals, learning task case studies, explainability and challenges. Neurocomputing, 404, 93-107.
Contractive AE
Alternativas: denoising, regularización embedding, manifold learning, t-SNE
CPU Activity (21 variables)
Satellite Image (36 variables)
Denoising AE
Alternativas: robust AE
Imágenes | MSE | Reducción |
---|---|---|
Originales | 0 | 100% |
Ruidosas | 1656.08 | 0% |
Basic AE | 576.68 | 62.14% |
Denoising | 159.74 | 88.94% |
AE con regularización de ruido gaussiano
Se detecta anomalía cuando el error de reconstrucción supera un umbral.
Se interpola en el espacio de codificación, generando siempre datos semánticamente válidos.
AE variacional
Alternativas: AE adversarial
1Chen, R. T., Li, X., Grosse, R. B., & Duvenaud, D. K. (2018). Isolating sources of disentanglement in variational autoencoders. Advances in neural information processing systems, 31.
Charte, D., Herrera, F., & Charte, F. (2019). Ruta: Implementations of neural autoencoders in R. Knowledge-Based Systems, 174, 4-8.
ruta_autoencoder
: modelo de aprendizaje ruta_network
: estructura de capas neuronales ruta_loss
: función a optimizar (interna a Keras o externa: correntropía)ruta_noise
: un tipo de ruido aplicable a datos de entradaencoded <- iris[, 1:4] %>% as.matrix() %>% autoencode(2, "robust")
learner <- autoencoder_sparse(
input() + dense(3, "tanh") + output(),
"mean_squared_error"
)
model <- train(learner, train_x, epochs = 200)
reconstructions <- reconstruct(model, test_x)
evaluate_mean_squared_error(model, test_x)
network <-
input() +
dense(256, "elu") +
variational_block(10, seed = 42) +
dense(256, "elu") +
output("sigmoid")
learner <- autoencoder_variational(network, loss = "binary_crossentropy")
model <- train(learner, x_train, epochs = 10)
samples <- model %>%
generate(dimensions = c(8, 5), side = 6, fixed_values = 0.99)
plot_matrix(samples)
Charte, D., Charte, F., García, S., & Herrera, F. (2019). A snapshot on nonstandard supervised learning problems: taxonomy, relationships, problem transformations and algorithm adaptations. Progress in Artificial Intelligence, 8(1), 1-14.
Salidas | Salida sin orden | Salida ordenada | ||||
---|---|---|---|---|---|---|
Escalar | Múltiple | Escalar | Múltiple | |||
Entradas | Discreta | Continua | Discreta | Continua | ||
Muestras sin orden | clas. estándar | clas. multietiqueta/ multi-dimensional | regr. ordinal | regr. estándar | clas. multietiqueta gradada | regr. multi-objetivo |
Relación de orden | - | - | clas. monotónica | regr. isotónica | - | - |
Múltiples instancias | clas. multi-instancia | clas. MIML/MIMD | - | regr. multi-instancia | - | - |
Múltiples vistas | clas. multi-vista | clas. MVML/MVMD | - | regr. multi-vista | - | - |
Tarea | Transformación del problema | Adaptación de algoritmos |
---|---|---|
MI | Embedded-space | SVM Neural networks k-NN |
MV | Canonical correlation analysis | SVM Fisher discriminant analysis |
ML | Binary Relevance Label Powerset Classifier chains | k-NN Decision trees SVM Association rules Ensembles |
MD | Independent classifiers | Bayesian networks Maximum Entropy |
LDL | Multiclass reduction | k-NN Neural networks |
LR | Pairwise preferences Constraint classification | Boosting SVM Perceptron |
MTR | ML-inspired: one-vs-all, stacking, regressor chains Support vectors | Generalizations Support vector regression Kernel-based Regression trees Random forests |
OR | Ordered partitions One-vs-next, One-vs-followers, One-vs-previous 3-class problems | Neural networks Extreme learning machines Decision trees Gaussian processes AdaBoost |
MC | Reduction to IR | k-NN Decision trees Decision rules Neural networks |
Charte, D., Charte, F., & Herrera, F. (2021). Reducing data complexity using autoencoders with class-informed loss functions. IEEE Transactions on Pattern Analysis and Machine Intelligence.
$$\Omega(\theta;S)=- \sum_{j=1}^{d}\left[P_+(j)~ \log\frac{P_+(j)}{P_-(j)} + H(j)\right]$$
donde \(P_+(j)\) indica la probabilidad de que la \(j\)-ésima variable tenga el valor más alto en las instancias positivas (y \(P_-(j)\) lo análogo para negativas):
$$ P_+(j)=\operatorname{softmax}\left(\frac{1}{N^{+}}\sum_{(x,+1)\in S}f(x)\right)(j)~,\quad P_-(j)=\operatorname{softmax}\left(\frac{1}{N^{-}}\sum_{(x,-1)\in S}f(x)\right)(j)$$
$$H(j)=-\left(P(j=0)~\log(P(j=0)) + P(j=1)~\log(P(j=1))\right) $$
PCA | LLE | Isomap | AE | Skaler | Scorer | Slicer | ||
---|---|---|---|---|---|---|---|---|
F1 | \(\otimes\) 5.885 | \(\otimes\) 6.115 | \(\otimes\) 5.731 | \(\otimes\) 4.038 | \(\bigstar\) 1.577 | 2.308 | 2.346 | |
F3 | \(\otimes\) 4.250 | \(\otimes\) 5.231 | \(\otimes\) 4.846 | \(\otimes\) 5.654 | 2.788 | 3.000 | \(\bigstar\) 2.231 | |
N3 | \(\otimes\) 4.692 | \(\otimes\) 5.173 | \(\otimes\) 5.308 | \(\times\) 4.269 | 3.654 | 2.577 | \(\bigstar\) 2.327 | |
L2 | \(\bigstar\) 2.712 | \(\times\) 4.519 | 3.981 | \(\otimes\) 5.654 | \(\times\) 4.442 | 3.846 | 2.846 | |
L3 | 2.769 | \(\times\) 4.500 | 4.115 | \(\otimes\) 5.500 | \(\otimes\) 4.923 | 3.654 | \(\bigstar\) 2.538 | |
\(\textit{ONB}_{\text{tot}}\) | \(\otimes\) 4.481 | \(\otimes\) 6.115 | \(\otimes\) 4.519 | \(\otimes\) 4.481 | 3.442 | 3.019 | \(\bigstar\) 1.942 | |
\(\textit{ONB}_{\text{avg}}\) | \(\otimes\) 4.442 | \(\otimes\) 6.077 | \(\otimes\) 4.519 | \(\otimes\) 4.577 | \(\times\) 3.538 | 2.904 | \(\bigstar\) 1.942 | |
kNN | F-score | 3.096 | \(\otimes\) 6.923 | \(\otimes\) 4.904 | 4.115 | 3.519 | 3.231 | \(\bigstar\) 2.212 |
AUC | 3.288 | \(\otimes\) 7.000 | \(\otimes\) 4.750 | \(\times\) 4.038 | 3.500 | 3.250 | \(\bigstar\) 2.173 | |
Kappa | 3.096 | \(\otimes\) 7.000 | \(\otimes\) 4.827 | \(\times\) 4.038 | 3.558 | 3.346 | \(\bigstar\) 2.135 | |
SVM | F-score | 3.788 | \(\otimes\) 6.846 | \(\otimes\) 4.673 | 3.538 | 3.538 | 2.923 | \(\bigstar\) 2.692 |
AUC | \(\times\) 4.000 | \(\otimes\) 6.846 | \(\otimes\) 4.865 | 3.462 | 3.346 | 2.962 | \(\bigstar\) 2.519 | |
Kappa | 3.904 | \(\otimes\) 6.846 | \(\otimes\) 4.827 | 3.346 | 3.577 | 2.962 | \(\bigstar\) 2.538 | |
MLP | F-score | \(\otimes\) 4.077 | \(\otimes\) 6.769 | \(\otimes\) 3.962 | \(\otimes\) 4.731 | 2.596 | \(\otimes\) 3.865 | \(\bigstar\) 2.000 |
AUC | \(\otimes\) 4.000 | \(\otimes\) 7.000 | \(\otimes\) 4.058 | \(\otimes\) 4.635 | 2.404 | \(\otimes\) 3.942 | \(\bigstar\) 1.962 | |
Kappa | \(\otimes\) 4.000 | \(\otimes\) 7.000 | \(\otimes\) 4.077 | \(\otimes\) 4.596 | 2.558 | \(\otimes\) 3.827 | \(\bigstar\) 1.942 | |
wins | 62 | 10 | 17 | 19 | 78 | 64 | 188 |
Tesis para la obtención del título de doctor por la Universidad de Granada
Programa de Doctorado en Tecnologías de la
Información y la Comunicación
Francisco David Charte Luque <fdavidcl@ugr.es>
Directores:
Francisco Herrera Triguero
Francisco Charte Ojeda
6 de julio de 2022