[ Python ] TensorFlow Weight L2, L1 Normalization 쉽게하기

https://stackoverflow.com/questions/38286717/tensorflow-regularization-with-l2-loss-how-to-apply-to-all-weights-not-just

TensorFlow - regularization with L2 loss, how to apply to all weights, not just last one?

https://stackoverflow.com/questions/36706379/how-to-exactly-add-l1-regularisation-to-tensorflow-error-function

How to exactly add L1 regularisation to tensorflow error function

기존 Loss gen_loss , disc_loss에 Weight만 L2 부여하기

t_vars = tf.trainable_variables()

G_L2 = []
for v in t_vars :
    if re.search('Weight' , v.name) :
        if re.search("Generator", v.name) :
            print(v.name)
            G_L2.append(tf.nn.l2_loss(v))
gen_loss += tf.add_n(G_L2)  * 0.001

D_L2 = []
for v in t_vars :
    if re.search('Discriminator' , v.name) :
        if re.search("kernel", v.name) :
            print(v.name)
            D_L2.append(tf.nn.l2_loss(v))
disc_loss += tf.add_n(D_L2)  * 0.001

L2, L1 Loss 해보기

vars = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES,scope="Ensembles")
L2 = []
WEIGHTS = []
import re 
for v in vars :
    if re.search('Weight' , v.name) :
        WEIGHTS.append(v)
        L2.append(tf.nn.l2_loss(v))
Loss += tf.add_n(L2)  * 0.1
l1_regularizer = tf.contrib.layers.l1_regularizer(scale=0.005, scope=None)
regularization_penalty = tf.contrib.layers.apply_regularization(l1_regularizer, WEIGHTS )
Loss += regularization_penalty